Connect With Us


Darpa picks Intel for big data and machine learning

Steve Rogerson
June 8, 2017
Intel has been selected by US defence agency Darpa to collaborate on the development of a data-handling and computing platform that will leverage machine learning and other artificial intelligence techniques.
The notion of big data emerges from the observation that 90 per cent of the data available today have been created in just the past two years. From devices at the edge to large data centres crunching everything from corporate clouds to future energy technology simulations, the world is awash in data being stored, indexed and accessed.
Darpa's microsystems technology office created the Hierarchical Identify Verify & Exploit (Hive) programme to develop new technologies to realise a thousand times performance-per-watt gains in the ability to handle graph analytics.
Unlike traditional analytics that are tools to study one-to-one or one-to-many relationships, graph analytics can use algorithms to construct and process the world's data organised in a many-to-many relationship, moving from immediate connections to multiple layers of indirect relationships. While some graphs are small and easy to visualise – such as a family tree – many graphs are vast and constantly changing, and they represent significant complex semantics, such as the evolving search list of every user on the planet for Amazon sales or Apple iTunes.
Intel's data centre and platform engineering groups and Intel Labs will work as one of the hardware architecture research performers for Darpa Hive, with a joint research programme between Intel and Darpa valued at more than $100m during a four and a half year effort.
"By mid-2021, the goal of Hive is to provide a 16-node demonstration platform showcasing 1000x performance-per-watt improvement over today's best-in-class hardware and software for graph analytics workloads," said Dhiraj Mallick, vice president of the data centre group at Intel. "Intel's interest and focus in the area may lead to earlier commercial products featuring components of this path-finding technology much sooner."