As tech-gear comes ever closer to the physical boundaries of electric-speed, hitherto-unseen bottlenecks are surfacing. Storage vendors have been selling systems using SSDs (solid-state drives, essentially large-capacity flash memory units with higher throughput speeds) at critical data-junctures for a couple of years now. As SSD prices continue to drop, expect this trend to speed up.
Speed is found across the tech-spectrum nowadays. Processors ramp up their clock-speeds and multi-core processors amplify dataflow. It all leads to more data.
Flash memory is now used to improve the performance of many devices and applications. Gartner VP David Cearley expects to see a huge jump in the use of flash memory in consumer devices, entertainment devices, equipment and other embedded IT systems.
"In addition, flash offers a new layer of the memory hierarchy in servers and client computers that has key advantages - space, heat, performance and ruggedness among them," said Cearley. "Unlike RAM, the main memory in servers and PCs, flash memory is persistent even when power is removed. In that way, it looks more like disk drives where we place information that must survive power-downs and reboots, yet it has much of the speed of memory, far faster than a disk drive."
Cearley added that software is critical to unlock these advantages. "As lower-cost - and lower-quality - flash is used in the data center, software that can optimize the use of flash and minimize the endurance cycles becomes critical," he said. "Users and IT providers should look at in-memory computing as a long-term technology trend that could have a disruptive impact comparable to that of cloud computing."
In-memory as a technology has come into focus over the last two to three years. However, there is nothing new about it, said Surya Mukherjee, lead analyst for Ovum's information management team. "Any memory faster than a disk drive, which has moving parts, will speed the process of data storage, retrieval and analysis," he said.
"As RAM and SSDs become cheaper, it's now possible for organizations to load an entire database onto fast memory and benefit from low-latency transactions," Mukherjee explained. "The popularity of in-memory analytics is closely linked to advances in hardware technology, such as 64-bit computing, multi-core processor, and improvements in processor speed. Technical advancements in these areas help vendors optimize the use of memory and speed up data processing performance."
Mukherjee says that "software applications using the memory should be optimized to deliver desired features within the application-tier and work with the data in-memory. Also the compression used within in-memory databases and applications should be optimized to perform read/write without the need for significant decompression of data."
Which vendors are racing to bring their in-memory products to market?
In December 2010 SAP launched HANA: software that uses an in-memory computing engine that allows data to be held in RAM instead of being read from disks or flash storage, thus providing a performance boost. SAP intends for HANA boxes to be attached to its own ERP systems, sucking in and analyzing transactional data in real time. However, HANA's agnostic data access functionality means any information source can be used.
In a research note last October, Gartner's Massimo Pezzini and Daniel Sholler wrote: "SAP has started a new generation of application infrastructure and architecture efforts focused on cloud and in-memory. The new vision will force the competition to respond; but addressing its potentially disruptive impact on SAP's most-conservative customers' requirements will be a key challenge."