Don't ever replace your battery or the power supply with one that it is not made for it--this is a very bad idea for a variety of reasons.
Basics on all batteries -
Battery voltage **rating **definitely should not change, be very steady, and match the original +- 2-3%. This opinion differs greatly from this post, however my opinion is just based on not knowing the specifics of that laptop's circuitry: "Laptop battery: is voltage really important to respect?" The number of amps that battery will deliver at that same voltage must also be sufficient, which is called power or wattage, (Amps x Volts = Watts). However, the amount of power you use with heavy use will deplete the battery much faster, and that battery capacity is rated in Amp/Hours. A/H capacity is reduced each time your discharge / charge that battery. Laptop batteries will generally only hold less than ~50% of their new capacity after 100 full cycles.
Laptop batteries -
Make SURE it is approved for YOUR laptop. There's a computer inside that laptop's battery, and it not only controls the health and charging of batteries inside, it also communicates with your PC. Most don't, but OEMs could prevent any non-OEM approved battery from even functioning. The amount of charge, and the rate of charge, is also regulated in most cases, by your laptop. Remember, that brick power supply also must feed the laptop itself, without the battery in most cases. (Don't run a laptop without a working battery, which acts to protect it from power surges better than any surge protector could.) OEM batteries that will operate perfectly, replacing the originals are extremely cheap and work great, if obtained from a reputable source. These have generally had the cells replaced with fresh ones, using the same circuitry. So, look on Amazon for your laptop's specific make and model, and read the reviews for potential problems. Also keep in mind that some Amazon vendors will sell "identical make and model" to what you are seeing, and they are not the same at all--YMMV.
Power supplies -
Extra or replacement power supplies are also extremely cheap these days. It is best to buy at least one extra, in case your original dies, which they do occasionally. Plus, if you commute, why carry, plug and unplug that brick a few times a day. Best is to keep a separate one ready, wherever you regularly use any laptop. They will work great, provided you get one your laptop recognizes as OEM for that laptop, voltage is the same, and the amps it will deliver is the same, or greater. If the laptop doesn't recognize the proper power supply, even with the correct voltage and greater amperage, it might run the laptop, but won't allow it to charge. Often, there's an extra wire to provide the battery-to-laptop communication link.
Role of internal support circuitry and software -
Additional circuitry inside laptops could make or break the requirement for tight voltage requirements, or any requirements. For instance, you want 120vAC? Then, take a few 12vDC batteries, add some circuitry, and wallah, you've got your 120vAC. the other significant role of the software is to ensure that whatever you are plugging into it, will work properly, or it could disable it completely if either not compatible or it was malfunctioning.
Laptops could be designed to take any battery, which however, would add to the cost, complexity, and unintended consequences. Even now, many OEMs, by design, still charge the batteries to 100%, showing both longer run times, but unnecessarily, MUCH shorter lifespans for lithium battery packs. Same problem for most all of our our cell phones, which are designed for the best advertised run time, not battery life.
If you don't have the right expertise in electronics, computer hardware, and battery systems, you stand a good chance of destroying your laptop, if you modify any of these. Worse, with the wrong modifications, you could literally blow up your battery, or at least catch it on fire. Most battery repair shops open to the walk-in public will never even try to replace faulty cells in a lithium battery pack. It is just too dangerous to do correctly, unless you know that specific battery pack. Check out the Youtube videos of lithium battery fires should convince you.