I met @sakacc last Thursday’s evening face-to-face enjoying a not so short chat that make me to re-think some given statements – doesn’t mean I’ve changed my thoughts about #Ceph and its assortment within type 4 , ok? –
I know that I’ve committed to write in my previous note about other features on #Ceph, but I will use this current space to talk about something more important, my internal new point of view about the future of the hardware: commodity is not the only option for the future deployment, that hardware could still bring a big different through innovation for some use cases.
Things like the DSSD’s acquisition: Why have they got other high performance storage technology if they already have XtreamIO? Well, the thing is they are not the same hardware storage technology. DSSD is aimed to cover needs for a much extreme low latency IO thanks to the hardware features and its internal architecture. It’s a complete new dimension that brings an awesome storage bandwidth with a bunch of silicon pieces bringing much less latency though a cool escalation mode.
If I’ve understood @sakacc well, this solution helps you to connect an array of memory storage directly through HDFS, it’s like a full mesh architecture interconnecting memory modules through some internal connection with a much lower latency than PCI or IB. Also, the memory is non-volatile, that means we don’t need to use any sort of disk including SSD to keep data persistency in case of any failure. An Hana’s scale-out cluster architecture requires fast disks to keep the data persistency in order to replace any server node in case of its failure, but this condition holds back the high performance you should directly get from memory as a trade-off to get high availability for your analytics service. Anyway, I think SAP and EMC are seeing to work together to run Hana on DSSD in the mid-term.
DSSD is not only the hardware technology, apparently they brings also a very efficient and solid low level code that brings a some special RAID management along the memory array. DSSD is not using any kind of Operating System, apparently DSSD is using a reduced piece of code that brings HDFS to the external world and it doesn’t use any known storage protocol at all to manage the internal data transit among storage devices.
Will DSSD break down the traditional way to implement databases?, Will the “in-memory” DB be the only reality in a few years? Will DSSD cannibalize an important part of the enterprise storage business?
I strongly suggest to read at “A new, 5th Branch in the Storage Tree of Life” by @sakacc that explain better the big difference in human time between nano-second and some milliseconds.
below you can see the picture about the possible future strategy at EMC for HDFS and its related products (courtesy by @sakacc)
See you next and Good Luck Chile this Saturday!