Selection of Crowd in Crowdsourcing for Smart Intelligent Applications: A Systematic Mapping Study

Table 9

Existing technique used for crowd selection.

S. no.	Description	Existing techniques	Citation
1	Workers are selected according to personal profiles that contain various attributes such as sex, age, workers’ ability, demography, expertise, experience, past performance, etc.	Profiling-based selection	[7, 39, 40, 42, 47, 49, 52, 57, 59, 68, 69, 71, 76, 81, 82]
2	For accomplishment of tasks, crowds are selected according to their skills related to various tasks. Skills test assessments are carried out to certify that workers have the necessary skills for completing tasks.	Skill assessments	[3, 7, 25, 31, 47, 49, 51, 52, 60, 76, 83]
3	Expertise-based selection is carried out to capture experts to perform specific tasks.	Expertise filtering	[3, 9, 13, 31, 46, 49, 58, 68, 71, 79]
4	Behavior and attributes are important characteristics of workers participating in various tasks. Workers are selected on the basis of their attitude and behaviors observed in performing various tasks.	Selection on the basis of crowd attributes, behaviors, and attitudes	[26, 31, 34, 36, 53, 57, 68]
5	Trust is an important factor for the selection of the appropriate worker for tasks. Workers are assigned with different trust values and they are selected based on these trust values.	Selection on the basis of trustworthiness	[3, 9, 29, 31, 34, 47, 48]
6	Individuals are selected based on their previous or current performance in different types of tasks.	Performance	[6, 36, 42, 47, 49, 56, 68, 81, 82]
7	Feedback is provided to rank workers for competing or quitting various tasks; crowds are selected on the basis of various feedback provided by third parties, i.e., organizations or other participating bodies.	Feedback mechanism	[3, 68]
8	Workers are selected on the basis of their roles.	Role-wise selection	[3, 48]
9	Workers are allowed for making their bid, and appropriate workers are selected according to their bid level to perform a task. The bids are associated with specific rewards.	Selection on the basis of bid	[3, 36, 50, 52]
10	Organization conducts qualification- or job-specific tests to assess and select workers for various types of tasks. Qualification test ensures that workers have relevant knowledge for the task.	Selection based on job/qualification tests	[3, 4, 6, 31, 66, 68]
11	Optimal set of workers are selected based on time and cost constraints. Workers are selected on the basis of timeline followed in previous completed tasks.	Time and constraints	[1, 3, 9, 40, 50]
12	Workers are grouped in registration order into five different belts, i.e., red, green, yellow, blue, and gray, that represent the skill level of workers.	Selection on the basis of belt level	[44]
13	Crowd participants are selected on the basis of their experience level.	Crowd selection on the basis of experience level	[7, 22, 24, 45, 46]
14	For worker selection, ranking is applied. Workers are classified into three categories, i.e., new comers, associates, and seniors. Ranking allows platforms to automatically find out an appropriate worker for a specific task.	Ranking	[47]
15	In mobile crowd sensing, task assignment techniques are used to select crowd workers. The task assignments’ framework is used to capture participants who use sensors for obtaining or measuring specific data.	Task assignment technique	[62]
16	A worker is selected on the basis of his or ability and capability toward a task.	Crowd selection on the basis of capability/Ability	[1, 23, 66, 71]
17	Candidates are selected according to their country of origin.	Demographic filtering	[4, 6, 22, 42, 68]
18	Before allocation of task to workers, their availability on platforms is observed.	Crowd selection on the basis of availability of workers	[13, 36, 40, 44, 80]
19	For assessing the crowd capabilities, a recommendation system was utilized. Crowd workers must have access to the recommendation system to help them in finding related tasks according to their profiles.	Recommendation-based crowd selection	[10, 28, 39, 48, 49, 71]
20	Screening of the crowd is carried out based on skills that are required for completing a task. Workers are screened out from a diverse group by conducting tests, etc.	Screening-based crowd selection	[25, 80]
21	Crowd workers’ recruitment is restricted to only workers who have declared that they have a background to a specific task and if they possess a working history of at least two years.	Relevant background filtering for crowd selection	[24, 60]
22	The reputation system is used to select participants on the basis of their level of reputation.	Crowd selection on the basis of reputation	[9, 50, 51, 65]
23	The crowd is rated based on the quality of responses. Rating assists in selecting the appropriate crowd for tasks. Workers are selected by job seekers according to their rating.	Selection of crowd on the basis of rating	[25, 52]
24	An incentive mechanism was proposed for worker selection. Workers are selected statically; after bidding, winners are selected dynamically	Crowd selection on the basis of incentives/reward mechanisms	[13, 23, 50]
25	The task requester selects workers based on some criteria, such as majority voting, for estimating correct answers of a task.	Crowd selection on the basis of voting scores	[26, 71]
26	Social relationship among requestors and solvers is established and the task is given to friends of a friend; due to this, the reliability of task distribution is increased.	Crowd selection according to social relation of workers	[29, 83]
27	Matches complex tasks requiring multiple skills to appropriate workers. Workers are invited for making a bid for a task, and this bidding was based on price as well as worker profile.	Crowd selection on the basis of auction	[3, 52]
28	To select a crowd (testers) from a large pool of people, the technique was used. The tasks are matched with crowd and only the appropriate crowd is permitted.	Crowd selection on the basis of matching mechanism	[22, 31, 45]