Fujitsu Laboratories has developed a new stream processing architecture capable of adding or changing content in real time while processing large volumes of IoT data.
Dracena (Dynamically Reconfigurable asynchronous consistent event-processing architecture) can add or change content while processing IoT data without stopping.
With recent advances in IoT technologies, it is expected that many real-time services will be created to utilize the large volumes of data flowing into the cloud from various devices across factories, homes, and social infrastructure.
In the progression towards autonomous vehicles with connected cars, researchers are considering the analysis of the vast amounts of information, such as speed and location, generated from vehicles, which can then be presented to drivers, in the form of warnings, for example.
Stream processing technology - which is effective in the high-speed processing of these sorts of huge volumes of data - has issues in that because processing must be temporarily stopped when changing or adding processing content according to additions or improvements to services, the provision of services can be delayed.
Now, Fujitsu has developed a new stream processing architecture that automatically switches to a newly provided data processing program when a parallelized data processing job has been completed, by separating stream processing into data reception processing and actual data processing so that data reception processing and current data processing are not stopped.
As a result, in a simulation of the reception of a few dozen bytes of data per second from one million vehicles, Fujitsu has confirmed that this architecture is able to continue processing streaming data while adding or changing processing programs, with an average delay increase volumes of five milliseconds or less.
Fujitsu Laboratories is looking to commercialize this technology during fiscal 2018 on the Mobility IoT Platform, offered by Fujitsu Limited, and extend it to other industry areas.
Details of this technology were presented at DEIM2018 (the Forum on Data Engineering and Information Management) in Awara, Fukui Prefecture, Japan.