When two technologies come together, they can create some new and amazing things, like the phone and browser coming together to form a smartphone. Today, developers are applying AI's ability to discover patterns to large graph databases, which store information about relationships between data points. The two come together to form a powerful new tool called Graph Neural Networks (GNN).

What are Graph Neural Networks

Graph Neural Networks apply the predictive power of deep learning to rich data structures that describe objects and their relationships as points connected by lines in a graph. In Graph Neural Networks, data points called "nodes" are connected by lines called "edges", and various elements are expressed in mathematical form, which allows machine learning algorithms to make useful predictions at the node, edge, or entire graph level.

What can Graph Neural Networks do

More and more companies are using GNN to improve drug development, fraud detection, and recommendation systems. These and many other applications rely on finding patterns in relationships between data points. Researchers are exploring the use cases of GNN in computer graphics, cybersecurity, genomics, and materials science. Recently, a paper described how GNN can use graph-based traffic maps to improve predictions about arrival times.

Many scientific and industrial fields have stored valuable data in graph databases. Through deep learning, they can train prediction models to uncover novel insights from the graph.

Many scientific and industrial fields can be expressed in graph form

Amazon Web Services (AWS) Senior Chief Scientist George Karypis said in a lecture earlier this year: "GNN is the hottest field in deep learning research. More and more applications are using GNN to improve their performance." Many people agree. Stanford University Associate Professor Jure Leskovec said: "GNN is gaining more and more attention, they can flexibly establish complex relationship models, which traditional neural networks cannot do." He showed the following AI paper chart in his speech, which mentioned GNN.

Amazon said in 2017 that it was using GNN to detect fraud. In 2020, Amazon launched a public GNN service for external users to use for fraud detection, recommendation systems, etc. To maintain high customer trust, Amazon's search engine uses GNN to detect malicious sellers, buyers, and products. With NVIDIA GPU, the search engine can explore a graph with tens of millions of nodes and billions of edges, and shorten training time from 24 hours to 5 hours.

LinkedIn Senior Software Engineer Jaewon Yang said in another meeting that LinkedIn uses GNN to provide social recommendations and understand the relationship between a person's skills and their job position.

NVIDIA Distinguished Engineer Joe Eaton said: "GNN is a general-purpose tool, we develop some new GNN applications every year." Currently, Joe Eaton is leading the team to apply accelerated computing to GNN. He said "We haven't even touched the surface functionality of GNN." Another indication also shows people's interest in GNN - Leskovec's course video on GNN at Stanford University has exceeded 700,000 views.

So far, deep learning has mainly focused on images and texts. These structured data can be described as word sequences or pixel grids. In contrast, graphs are unstructured, so they can be any shape or size, and contain data of any type such as images and texts. GNN uses a process called information passing to organize the graph so that machine learning algorithms can be used. Information passing embeds information about adjacent nodes into each node. AI models use embedded information to find patterns and make predictions.

Data flow examples in three types of GNN

For example, recommendation systems use the way nodes are embedded in GNN to match customers and products; fraud detection systems use edge embedding to find suspicious transactions; drug discovery models find reactions between them by comparing entire molecular graphs.

GNN flow outputs prediction after input graph

The Italian researchers who published this paper in 2009 first named this neural network as "Graph Neural Networks". But it wasn't until eight years later that two researchers from Amsterdam used a variant of Graph Neural Networks called "Graph Convolutional Networks" (GCN) to show the power of this neural network. GCN is also one of the most popular GNNs today. GCN inspired Leskovec and his two Stanford University graduate students to create GraphSage - a GNN that shows a new way of working with information passing.

GraphSage has created a powerful aggregation technique for passing information in GNN

They created PinSage, a recommendation system with 30 billion nodes and 180 billion edges, which exceeded other AI models at the time. Today, Pinterest applies PinSage to more than 100 use cases in the company. The company's senior machine learning engineer Andrew Zhai said in the online meeting: "Without GNN, Pinterest wouldn't have the attraction it has today." At the same time, other variants and hybrids also appeared, including graph recurrent networks and graph attention networks. GAT uses the attention mechanism defined in the Transformer model to help GNN focus on the most relevant part of the data set.

Graph Neural Network Variation Family Tree Overview

Looking forward, GNN needs to be fully expanded. Enterprises that haven't maintained graph databases need to use tools to reduce the workload of creating these complex data structures. People who use graph databases know that in some cases these databases will continue to expand, and single nodes or edges will be embedded into thousands of features. This poses a challenge for efficiently loading massive data sets from sub-systems into processors.

As part of this work, NVIDIA announced at GTC that, in addition to Deep Graph Library (DGL), it now also supports PyTorch Geometric (PyG). These are the two most popular GNN software frameworks today. NVIDIA provides various tools to accelerate GNN construction

NVIDIA provides various tools to accelerate GNN construction

The NVIDIA-optimized DGL and PyG containers are performance-tuned and tested for NVIDIA GPUs. They provide convenient platforms for people to start developing GNN applications.