**Keyword [GSNN] [GGNN]**

Marino K, Salakhutdinov R, Gupta A. The more you know: Using knowledge graphs for image classification[J]. arXiv preprint arXiv:1612.04844, 2016.

# 1. Overview

## 1.1. Motivation

- human has ability to acquire knowledge about the world and use knowledge to reason about the visual world
**Gated Graph Neural Network**(GGNN) suffers computational issues when graph is large

In this paper

- proposed
**Graph Search Neural Network**(GSNN) mitigates the computational issues - using knowledge improves performance on image classification

## 1.2. Contribution

**GSNN**. incorporate potentially large knowledge graph into end-to-end system

# 2. Methods

## 2.1. GGNN

**input**. graph of N nodes**output**. every graph node or global output

**h_{v}^{t}**. hidden state for**node v**at time**step t****x_v**. problem specific annotation**A_v**. adjacency matrix of graph for**node v****W, U**. learned parameters**(1)**. Initialization**(2)**. propagation updates from adjacent nodes**(3-6)**. combine the information from a**djacent nodes**and**current hidden state**of the nodes to compute the next hidden state

output network (g). FC network

## 2.2. GSNN

- propagation net
- importance net
- output net

- start with
**initial nodes**in the graph based on the likelihood of the concept being present (faster R-CNN) **add adjacent node in, learn a per-node score function**to estimates the importance of these nodes

**importance network**. assign target importance value to each node, correspond to gt is 1, neighbours are γ, two-hop away γ^2- add
**top P nodes**to**expanded**set - at final step T,
**compute the per-node-output and re-order and zero-pad the outputs into the final classification net**