This is a GREAT question and I find the responses fascinating. I am going to comment on this as an Oracle DBA and my answers are SPECIFIC to the Oracle database. This is a major mistake a lot of people make when working with Oracle. I am not sure if this applies to other applications as well. This is not meant to be off-topic, but is meant as a specialized answer.
When you tune performance with Oracle, you are really looking to clear up bottlenecks. Though most of us don't say it, it is based on the Theory of Constraints: https://en.wikipedia.org/wiki/Theory_of_constraints
Memory may not be your bottleneck. Oracle has complex mechanisms for managing memory and just increasing memory can actually slow things down if other areas are where the bottleneck is. Let me give you one example that is VERY common.
Queries seem to be slow. The consensus is if we increase RAM, we should increase the response time of queries since memory is faster than disc. Well... This is how Oracle handles memory management for data. Oracle has a variety of memory locations that are allocated to specific duties. So you can increase these memories. The area used for data is called the 'buffer cache'. This is a series of linked lists (the number of them tend to increase with each version). Every time a block is found on disc during a query, a hash algorithm is run on it to determine which list to stick it in. Where to put it in the list is based on a touch count algorithm (explained on the Oracle support site, so you have to pay to get it... it's not really important).
HOWEVER, when you run a query, Oracle takes out a latch on the buffer chain you search at the time. This LATCH (note: this is not a lock. Google "latch" if you don't know the difference) blocks all other operations on that chain for the duration of your read. So it blocks reads AND writes (this is entirely different than Oracle claiming locks don't block reads).
This is necessary because as you read the block in the chain, Oracle moves it around based on how often it is 'requested'. More frequently requested blocks are moved to the top and less frequently requested blocks are left at the bottom and aged out. You cannot have 2 sessions reading a linked list and moving blocks around or you will hit pointers that point to non-existent locations.
When you increase the size of memory, you increase the size of each linked list. This increases the time it takes to read the list. A single poor query or complex query can do tens of thousands or even millions of reads down linked lists. Each read is fast, but the number of them leads to latches taken and these will block other sessions. Oracle calls this a 'logical IO' (or buffer get or some other stuff. This lingo is specific to Oracle and may mean something else in other parts of IT).
So, if the list is longer and you have really bad SQL, then the SQL statements will hold their latches longer. Increasing memory can occasionally REDUCE performance. Most of the time, this won't happen. People will spend a lot of money and see no benefit. That being said, there are times when you need more memory in the buffer cache, but you have to properly identify the bottleneck to know whether this is appropriate. I can't discuss how to analyze this in this post. See the DBA forums. Some people discuss it there. It is rather complex.
Does anyone have specific examples with other pieces of software where this can happen? There is a terrific business book called 'The Goal' that discusses alleviating constraints in a factory. This process is very similar to what Oracle DBAs do when assessing performance issues. It is often standard reading in MBA programs. It is very valuable to read for IT professions.
https://en.wikipedia.org/wiki/Eliyahu_M._Goldratt