Optimizing Hadoop Performance for Big Data Analytics in Smart Grid

Table 6

GEP recommended configuration parameter settings.

Configuration parameters	Optimized values
Number of data samples in million	17.28	34.56	51.84	69.12	86.40
io.sort.factor	10	10	13	19	12
io.sort.mb	38	13	58	62	55
io.sort.spill.percent	0.90	0.90	0.90	0.89	0.90
mapred.reduce.tasks	14	14	13	2	16
mapreduce.tasktracker.map.tasks.maximum	8	6	8	8	8
mapreduce.tasktracker.reduce.tasks.maximum	1	1	1	2	1
mapred.child.java.opts	169	135	121	124	121
mapreduce.reduce.shuffle.input.buffer.percent	0.73	0.65	0.65	0.65	0.75
mapred.inmem.merge.threshold	200	200	201	202	201