Well theres some logic here thats outside the boundaries of reality. It a actually very common in our language, take heat and cold, cold does not exist, its just less hot heat. Because of language it is thus granted that you could transfer cold into hot, however in realty you can only transfer heat to cold, but not the other way around. The difference although subtle has many deep implications (such as why a refrigerator unit is a risking a fire in your apartment, because its moving hotter to colder, it just cools the pipes before it goes in to the refrigerator unit to again transfer the hot air in the refrigerator to the pipes, which then get heated up to push it out in the back)
Here you have the image plane versus nothing in your thinking, ok so how does nothing convey in 3d its the background color. Why, because a image cant have pixels with no color. You do however have tool in the toolkit that can kindof do this, its just invisible in the render view. Its the alpha channel, it is possible to tag the back nothing as trasparent with alpha. But its still has color, it will allwas have color, in in case of black absence of it, but we call it a color because we can precieve it as much as we can red.