First, I have to say that swap/pagefile isn't RAM or virtual RAM. It's a BIG misconception that lots of people have. Pagefile is simply a place to store "unused" memory pages. If a program touch a page that's not available in memory then the memory controller will throw an exception that the OS will catch and load that page from any kind of memory like hard disk, SSD, network or zram, zcache... or simply kill the process away. That no way means pagefile is RAM
Now back to the main question
Indeed SSDs are always slower than RAM in the same era, but...
The reason isn't speed!!!
Even if SSD is faster than RAM we still can't use it as RAM, because the SSDs you can buy nowadays doesn't have the capability to address individual bytes like RAM. They're block devices which means you can only work on blocks. Under the hood they're comprised of NAND flash modules which can only read/write single pages. You can't edit a single byte without reading and rewriting the whole page. Each page is far bigger than the CPU's cache line
That means unlike what psusi said in the second paragraph, an NVMe or M.2 SSD can't work as RAM even if you can map it to the CPU's address space!!!
There are some kinds of non-volatile memory that's byte-addressable though, for example NOR flash. That's why small microcontrollers for embedded devices occasionally use NOR flash as an execute in place (XIP) medium to avoid the need for separate RAM and EEPROM/flash chips. But that doesn't work for large computers because of the speed and cost, just like how some small MCUs use SRAM directly as RAM instead of DRAM like normal CPUs
Additionally, to be able to work as RAM you not only need the ability to read/write single bytes but also the support from the memory controller/MMU. Mapping the address space doesn't work if the memory controller doesn't know how to interact with the device. Currently memory controllers all deal with RAM modules directly, so you need to use the same RAM interface for anything intended to be used as RAM regardless of the memory type
That's what Intel® Optane™ A.K.A 3D XPoint is currently doing. It's currently the closest to "using SSD as RAM" because it's the only thing that's fast enough and also byte-addressable. Note that only the Intel® Optane™ Persistent Memory comes in the DDR4 form factor and can be used as RAM. You still need some special hardware and OS support though, otherwise they still assume data in RAM are always volatile

The other form factors, Intel® Optane™ SSD, can't be used as RAM due to the reasons I said
Intel® Optane™ memory comes in the M.2 form factor and will not be used in the DIMM/ DRAM memory slots. As such, it will not replace the current RAM that you have in place.
Can Intel® Optane™ Memory be used as RAM?
The price isn't cheap though, as expected: Intel Optane DIMM Pricing: $695 for 128GB, $2595 for 256GB, $7816 for 512GB (April 07, 2019)
See also