I would like to offer an alternative to David Postill's well thought out answer. In his answer, he approached the question of pixels being square, just as the title suggested. However, he made a very insightful comment in his answer:
Some would argue that they are never square ("A pixel is a point sample. It exists only at a point.").
This position can actually spawn off a whole different answer. Instead of focusing on why each pixel is a square (or not), it can focus on why we tend to organize these point-samplings into rectangular grids. It actually wasn't always that way!
To make this argument, we're going to play back and forth between treating an image as abstract data (such as a grid of points), and the implementation thereof in hardware. Sometimes one view is more meaningful than the other.
To start, let's go quite far back. Traditional film photography had no "grid" at all, which is one reason why the pictures always looked so crisp compared to modern digital ones. Instead, it had a "grain" which was a random distribution of crystals on the film. It was roughly uniform, but it was not a nice rectilinear array. The organization of these grains arose from the production process of the film, using chemical properties. As a result, film really didn't have a "direction" to it. It was just a 2d spattering of information.
Fast forward to the TV, specifically the old scanning CRTs. CRTs needed something different from photos: they needed to be able to represent their content as data. In particular, it needed to be data that could stream, in analog, over a wire (typically as a continuously changing set of voltages). The photo was 2d, but we needed to turn it into a 1d structure so that it could just vary in one dimention (time). The solution was to slice the image up by lines (not pixels!). The image was encoded line by line. Each line was an analog stream of data, not a digital sampling, but the lines were separated from each other. Thus, the data was discrete in the vertical direction, but continuous in the horizontal direction.
TVs had to render this data using physical phosphors, and a color TV required a grid to divide them into pixels. Each TV could do this differently in the horizontal direction, offering more pixels or fewer pixels, but they had to have the same number of lines. In theory, they could have offset every other row of pixels, exactly as you suggest. However, in practice this wasn't needed. In fact, they went even further. It was quickly realized that the human eye handled movement in a way that let them actually only send half the image every frame! On one frame, they'd send the odd numbered lines, and on the next frame ,they'd send the even numbered lines, and stitch them together.
Since that time, digitizing these interlaced images has been a bit of a trick. If I had a 480 line image, I actually only have half the data in each frame due to interlacing. The result of this is very visible when you try to see something move fast across the screen: each line is temporally shifted 1 frame from the other, creating horizontal streaks in fast moving things. I mention this because it's rather amusing: your suggestion offsets every other row in the grid by half a pixel to the right, while interlacing shifts every other row in the grid by half in time!
Frankly, it is easier to make these nice rectangular grids for things. With no technical reason to do any better than that, it stuck. Then we hit the computer era. Computers needed to generate these video signals, but they had no analog capabilities to write out an analog line. The solution was natural, the data was split into pixels. Now the data was discrete in both vertical and horizontal. All that was left was to pick how to make the grid.
Making a rectangular grid was extremely natural. First off, every TV out there was already doing it! Second, the math for drawing lines on a rectangular grid is much simpler than drawing them on a hexagonal one. You might say "but you can draw smooth lines in 3 directions on a hexagonal grid, but only 2 in the rectangular one." However, rectangular grids made it easy to draw horizontal and vertical lines. Hexagonal grids can only be made to draw one or the other. In that era, not many people were using hexagonal shapes for any of their non-computing efforts (rectangular paper, rectangular doors, rectangular houses...). The ability to make smooth horizontal and vertical lines far outstripped the value of making smooth full color imagery... especially given that the first displays were monochrome and it would be a long time before smoothness of imagery played a major part in thinking.
From there, you have a very strong precedent for a rectangular grid. The graphics hardware supported what the software was doing (rectangular grids), and the software targeted the hardware (rectangular grids). In theory some hardware might have tried to make a hexagonal grid, but the software just didn't reward it, and nobody wanted to pay for twice as much hardware!
This fast forwards us to today. We still want nice smooth horizontal and vertical lines, but with high end retina displays, that's getting easier and easier. However, developers are still trained to think in terms of the old rectangular grid. We are seeing some new APIs support "logical coordinates" and doing anti-aliasing to make it seem like there's a full continuous 2d space to play with rather than a grid of rigid 2d pixels, but its slow. Eventually, we might see hexagonal grids.
We actually do see them, just not with screens. In print, it is very common to use a hexagonal grid. The human eye accepts the hexagonal grid much faster than it accepts a rectangular grid. It has to do with the way lines "alias" in the different systems. Hexagonal grids alias in a less harsh way, which the eye is more comfortable with (if a hex grid needs to go one row up or down, they get to do it smoothly over a diagonal transition. Rectangular grids have to skip, creating a very clear discontinuity)