Artificial Intelligence (AI) and Machine Learning (ML) technologies have recently made inroads into several domains, including agriculture, healthcare, transportation, finance, and commerce. It is estimated that AI technologies will contribute to about 1.2 per cent additional GDP growth per year, contributing to about USD 13 trillion worth of economic activity by the year 2030. To achieve these targets, many countries, including India, have drafted strategies for enabling large scale application of AI for societal and public use. As indicated in the strategy paper by NITI Aayog India in 2018, it is envisioned to create and develop a National AI Marketplace (NAIM). The NAIM is conceptualized to host a large number of appropriately annotated data sets that can be used by model builders and application developers to train their algorithms.  

A typical AI Marketplace is a platform that crowd-sources a large number of datasets to be transacted with the users. However, challenges arise since the data supplied by crowd workers can be prone to errors and omissions. Technologies and processes have been developed to handle such quality issues by assigning tasks to multiple workers and aggregating workers' responses with the help of algorithms. The economic literature on crowd-sourcing focuses on developing appropriate incentive mechanisms to elicit high-quality datasets. Since, in a geographically dispersed crowd-sourced marketplace, the consumers do not know producers in person, they can rely on the producer's reputation that can indicate community-wide judgment on a given worker's capabilities. To create trust among strangers, most online platforms use variants of reputation systems to record qualitative reviews and numerical ratings tied to the profile of a platform user. In our recent research, we have modeled such a marketplace in which the datasets created by the crowd workers are rated by the consumers. In turn, the crowd adjusts the quality of their data sets depending upon the ratings given by the consumers to change the quality of their data sets. 

From a regulatory perspective, reputation systems fulfill a role similar to more traditional means of market regulation. Therefore, some researchers claim that reputation systems are a way of creating 'self-policing communities' that make conventional forms of consumer regulation superfluous. However, economic studies suggest that there are some inherent weaknesses in the self-regulating model of online marketplaces. The concerns about the integrity of reputation mechanisms have recently prompted regulatory initiatives in some EU member states. The spectrum of regulations ranges from traditional instruments such as guidelines issued by national market watchdogs and legislative amendments to consumer laws to relatively novel tools such as standards drafted by national standardization bodies. Regulations can also vary from being monitoring-only passive measures to active ones wherein regulators penalize the producers if their data sets do not meet the prescribed minimum standard quality thresholds. Our models indicate that while active regulation improves in general, the quality of the data sets, they have certain limitations. We observe through simulations that the quality of the data sets in the marketplace deteriorates if the regulator imposes very stringent minimum quality thresholds. Producers in the marketplace often deviate from the set standards and produce lesser quality artefacts due to inappropriate penalties and improper incentives.  

Hence while the governments are actively looking at creating AI marketplaces, they should also be regulating them actively for effective and efficient functioning. We advocate light-touch regulation with active monitoring and appropriate rating of the datasets and producers in the marketplace so that they can conform to the minimum prescribed quality standards.  

This article is authored by V. Sridhar & Shrisha Rao; both faculty at the International Institute of Information Technology Bangalore, India; winners of the Facebook Ethics in AI Research award. 

Dr. V. Sridhar is Professor at the Centre for IT and Public Policy at the International Institute of Information Technology Bangalore, India. He is the author of two books published by the Oxford University Press: The Telecom Revolution in India: Technology, Regulation and Policy (2012), and The Dynamics of Spectrum Management: Legacy, Technology, and Economics (2014). His third book titled Emerging ICT Polices and Regulations: Roadmap to Digital Economies was published in 2019 by Springer Nature. He is currently editing a book titled Data Centric Living: Algorithms, Digitization and Regulation, to be published by Routledge. Dr. Sridhar has taught at many Institutions in the USA, Finland, New Zealand and India. He has received funding from different sources, both national and international for his research projects, recent ones being from Facebook and Azim Premji University Research Foundation. Dr. Sridhar has a Ph.D. from the University of Iowa, US.A. His work can be accessed at: http://www.vsridhar.info 

Dr. Shrisha Rao received his Ph.D. in computer science from the University of Iowa, and before that his M.S. in logic and computation from Carnegie Mellon University. He is a professor at IIIT-Bangalore, a graduate school of information technology in Bangalore, India. Dr. Rao is an ACM Distinguished Speaker and a Senior Member of the IEEE. He is also a life member of the American Mathematical Society and the Computer Society of India. 

His primary research interests are in artificial intelligence and agent-based modeling, including in bioinformatics and computational biology. He also works on algorithms and approaches for resource management in complex systems such as used in cloud computing, and also has interests in energy efficiency, computational sustainability, and intelligent transportation systems.