3D Deep Heterogeneous Manifold Network for Behavior Recognition

Algorithm 1

Graph construction method based on Riemannian similarity metric.

	Input: trajectory curves of all skeletons ; behavior sequence label in training set ; total number of behavior categories ;
(1)	for Given behavior category do
(2)	Calculate the average trajectory curve of each class on the manifold;
(3)	Average trajectory curve ;
(4)	end for
(5)	for all Training trajectory curve with label do
(6)	Continuously project training trajectory curve along the average trajectory curve ,
(7)	Obtain the curve features on the tangent space after continuous projection;
(8)	end for;
(9)	for all Training trajectory curve do
(10)	Given test set trajectory curve
(11)	for; ; do
(12)	Continuously project test set trajectory curve along the average trajectory curve ;
(13)	end for
(14)	Continuously unfold test set trajectory curve along the path of average curves, obtain a set of curves
(15)	Calculate the set of similarity scores between each curve in the curve set and the corresponding average curve ;
(16)	Obtain the features under the score reflecting to the highest similarity;
(17)	end for;
(18)	for all Training trajectory curve do
(19)	Given a curve feature, use DTW to calculate the most similar trajectory curve to this curve;
(20)	Get adjacency list ;
(21)	end for;
(22)	for all Test track curves do
(23)	Given a curve feature, use DTW to calculate the most similar trajectory curve to this curve;
(24)	Get adjacency list ;
(25)	end for;
	Output: Curve features of training set and test set after continuous projection; The adjacency list obtained of the training set and test set ;