Actually, the CPU seems to make some difference. I had a AMD Athlon 64 3.0 GHz and upgraded to 3.5, after which almost all the default settings got a little higher. The max number of visitors also went up from 2 to 6 I think. I don't remember for sure since I use the cheat to up them.
There also is less lagging when I stubbornly continue playing large lots with loads of objects and many sims...
As for memory, I later upgraded from 1GB to 2 and didn't notice much difference. Up to 1GB the difference is really big though.