At some point both higher latencies and deeper cache hierarchies seem inevitable: http://www.ilikebigbits.com/2014_04_21_myth_of_ram_1.html

Though, perhaps 3D integration can stave that off for many more generations.

Interestingly, Broadwell-S and Broadwell-R did this using the "Crystal Well" L4 cache.

However, DDR4 memory now provides more bandwidth than the L4 could (although probably at a higher latency).