Short summary
The 12-cell battery will be fine. A difference of only 0.3V over 10.8 V will make no difference. Your laptop is already handling far wider variations in battery voltage than that.
Details
Your battery uses Li-ion cells, likely 18650 cells. If you look these up from component suppliers online you will find that some of these have a rated terminal voltage of 3.6 V, some 3.7 V. Three of those in series get you to 10.8 V or 11.1 V, respectively. So the difference in rated voltage is simply due to the choice of cells.
What really clinches the "it'll be fine" claim is this: Li-ion cells do not provide anything like a constant voltage anyway! When fully charged they are at 4 to 4.2 V/cell; voltage drops considerably as they are discharged; they are usually considered "fully discharged" at 3.0 V/cell or a bit lower. So your laptop has already been coping with input voltage of around 12 V, when the battery is freshly charged!
This works because the first thing that happens when current enters your laptop from the battery is that it's applied to a power controller circuit. This includes voltage regulators that use a "bucking" circuit to knock the voltage down to various voltages that the CPU, hard drive/SSD, screen, etc., need. These "end use" voltages will be considerably less than 10 volts - typically 5 volts or less! The voltage regulators can tolerate a very wide range of inputs. 0.3 V increase over 10.8 V is less than a 3% change. That's nothing. Don't worry about it. Enjoy the extra runtime.
btw, a laptop battery of course uses paralleled cells to increase the capacity (amp-hours or milliamp-hours). They arrange them with two or more cells in parallel, then these parallel sets are put in series to get the required voltage. This is because during charging, each paralleled set (3.7 V nominal) is charged and monitored individually, so connections to each point in the series string are brought to the outside. That's why your laptop battery needs so many contacts.
So your battery must use three paralleled sets in series. The six-cell must have three sets of two cells each, and to get your quoted 4400 mAh capacity each cell must be rated at 2200 mAh. This is a typical (though conservative) rating for older 18650 cells. The twelve-cell must be using sets of four batteries, and one would think that this would simply provide 8800 mAh. But, again, if you look at the catalogs for 18650 cells you will find that they are available in a wide variety of rated capacities. The numbers say that the 12-cell pack must be using cells rated at around 3400 mAh, which is not at all untypical, especially for more modern cells.
