Chapter 15: Artificial Intelligence, Big Data, and Emerging Risks

Introduction

Artificial Intelligence (AI) and Big Data are now key technologies of today's digital economy, changing the way information is captured, analyzed, understood, and applied. These technologies allow governments, corporations, research institutions and international organisations to handle vast amounts of data at an unprecedented speed and intensity. AI systems can detect patterns, automate complex tasks, make predictions, assist with decision making and constantly improve their performance using machine learning processes, through the use of advanced computational methods. At the same time, Big Data technologies enable organizations to manage and analyse huge amounts of data produced from digital platforms, social media, mobile devices, cloud infrastructures, sensors and connected technologies. AI and Big Data are a formidable combination that has the potential to transform the world by fostering innovation, productivity, economic growth and digital transformation (OECD, 2023).

AI and the rise of Big Data is increasingly playing a crucial role in practically every area of humanity. In the medical field, AI systems help in disease diagnosis, medical imaging analysis, personalized treatment plans, and drug discovery. In the world of finance, machine learning algorithms can be used in fraud detection, risk assessment, credit scoring, and automated trading. AI technologies are being increasingly adopted by governments to enhance public service delivery, resource allocation and planning of policies. From education to business, both sectors are turning to data analytics to drive improvements in learning outcomes and predictive analytics to optimize operations, deliver better customer experiences, and stay competitive. The above applications illustrate how vast the possibilities of AI and Big Data are to tackle societal challenges and enhance efficiency within organizations (World Economic Forum, 2024).

While the potential of AI and Big Data offers great promise, it has raised a number of concerns about privacy, ethics, governance, accountability, and human rights. With the volumes of personal data being gathered and processed by organizations, questions are raised on how it is gathered, stored, shared, and used. The centralisation of huge amounts of data in government and private companies has raised such issues as surveillance, lack of privacy and too much control over personal information. Moreover, AI systems can have a significant impact on people's lives, raising questions of fairness, discrimination, accountability, and transparency (UNCTAD, 2024).

One of the key features of today's AI systems is that they rely on data. Machine learning algorithms need large amounts of data to be trained, to recognize patterns and to make more accurate predictions. Quality, quantity, and diversity of the data are, therefore, directly related to the effectiveness of AI systems. This relationship has earned data the nickname of “fuel” in the world of artificial intelligence. As people grow more and more reliant on data, however, there is also a tendency to collect and aggregate data, which can lead to issues of consent, purpose limitation, data minimisation and individual rights. However, the need for more data to improve AI performance often clashes with data protection principles (World Bank, 2023), leading to a conflict between technological progress and data protection values.

The use of AI in key industries brings opportunities — and dangers. In the medical field, AI systems can enhance patient results and minimize misdiagnosis, however, faulty algorithms can have negative results on clinical decisions. Automated systems can boost efficiency and widen access to credit in financial services, but if the algorithms are biased they can disadvantage specific groups. Predictive policing applications are tools that in the law enforcement sector can help prevent crime, but can also perpetuate social stigmas and have a disproportionate impact on marginalized populations. Likewise, AI-powered governance systems could help to enhance the efficiency of governance systems while also sparking concerns about transparency, accountability, and democratic oversight (Kshetri, 2022).

Algorithmic bias and discrimination are one of the biggest concerns related to AI use and big-data. AI systems learn from past data, and if the data demonstrates social inequities or bias, so do the algorithms themselves. The impact of biased AI systems can be on employment, lending, education, healthcare, insurance, and criminal justice. Even in cases where algorithms and systems are not purposefully designed to discriminate, the outcomes can be discriminatory, emphasizing the need to ensure the quality of data, assess fairness, and continuously monitor the performance of algorithms and systems (OECD, 2023).

There has also been an increased focus on privacy and surveillance issues. Both public and private entities have seen a proliferation in the use of facial recognition, biometric, location tracking, and behavioral monitoring technologies that increase their ability to surveille and monitor individual actions. Such technologies can be used to enhance public safety, security and service delivery goals, but can also be of serious concern to civil liberties, freedom of expression and potential misuse of personal information. Securing privacy and maintaining balance between security interests and privacy rights is one of the most discussed topics in current digital governance.

Another difficulty that is coming is the use of predictive analytics. By leveraging past and present information, AI can predict future actions, likes, and results with high precision. In the fields of marketing, healthcare, finance, insurance and public administration, predictive models are becoming more and more common. While predictive analytics can enhance decision making and resource planning, there are also ethical concerns regarding profiling, automated decision making and the possibility of people being judged based on their predicted as opposed to their real behavior. These practices can have an impact on opportunities, access to services and autonomy in ways that are not always obvious or transparent or accounted for (World Bank, 2023).

With the advent of generative AI technologies, there are new risks, especially concerning deepfakes and synthetic media. Deepfakes are images, videos, or voices that are artificially generated, yet look and sound very real and can impersonate real people. As generative AI tools continue to improve, it's growing harder to tell the difference between real content and fake content. This creates a significant problem for information integrity, political stability, public trust, cyber security and reputation. The use of deepfakes could enable misinformation campaigns, scams, identity theft and public opinion manipulation, posing governance challenges for both regulators and technology companies (World Economic Forum, 2024).

Cybersecurity threats are also changing in tandem with AI and Big Data technologies. AI can enhance cybersecurity by bolstering automated threat detection and response systems, but malicious actors have also leveraged AI to create sophisticated cyberattacks, phishing campaigns, malware, and disinformation campaigns. As digital ecosystems become more complex, there is a greater attack surface for bad actors and managing risk and governance becomes more difficult. With the increasing availability of AI technologies, its misuse for criminal or fraudulent or malicious activities will likely grow in volume and cause concern.

With the rise of autonomous systems, the governance of AI is further complicated. The use of autonomous vehicles, robotic systems, intelligent infrastructure and automated decision-making platforms is increasingly taking place with limited human involvement. They pose issues of legal responsibility, accountability, safety, and ethics for these systems. Accountability for damage inflicted by autonomous systems is still a complex challenge and not adequately covered by current legal frameworks (UNCTAD, 2024).

Internationally, governments and rule makers are actively seeking solutions to AI governance and data regulation. Policy-makers are looking to "explain the balance between innovation and economic growth, and privacy, human rights, fairness and public trust. Yet, there are still differences between countries in the regulatory priorities, ethical standards and governance models. This regulatory patchwork poses difficulties for multinational organizations and makes it difficult to set uniform standards for AI and data governance on the international level.

With the ongoing development of AI and Big Data, the risks of these technologies are likely to get more intricate and widespread. Adaptive regulatory frameworks, increased institutional capability, international cooperation, ethical guidelines and continued public engagement will be essential to effective governance. In the digital era, technological innovation has to be in line with societal values and technologic threats are emerging that can jeopardize the trust, security and individual rights.

This chapter examines how artificial intelligence and data dependency are interconnected, as well as the threat of algorithmic bias, the use of facial recognition, predictive analytics and deep fakes, cybersecurity risks, and the changing nature of the risks facing advanced digital technologies. It is important to grasp these challenges in order to devise governance mechanisms that can address the opportunities and risks that can arise in an ever-changing digital economy.

15.1 AI and Data Dependency

Data is the lifeblood of Artificial Intelligence (AI) systems, being their core foundation for development, operation, and ongoing improvement. AI systems learn patterns, relationships, and behaviors from data, unlike traditional software programs which follow pre-defined instructions. This learning process allows AI technologies to carry out various functions, including image recognition, language translation, fraud detection, medical diagnosis, predictive analytics, and autonomous decision-making. Because of this, data is now being referred to as the “fuel” that drives modern AI systems, and is referred to as the “ingredient” of AI by some scholars and policy makers. The usefulness, reliability, and credibility of an AI system thus depend on the availability and quality of the data it is trained and used on (OECD, 2023).

A key trait of AI systems is the need for massive amounts of data. The modern machine learning models and deep learning models are very data-intensive to find patterns, recognising correlation and make accurate predictions. As the data volume and variety increase, so do the possibilities for AI systems to enhance their performance and accuracy. Healthcare AI systems, for instance, depend on a wealth of medical records, diagnoses, and clinical data to detect diseases and suggest treatments. In the same way, banks and other financial organizations analyze vast amounts of customer transaction data and fraud patterns to identify and prevent fraudulent transactions and to evaluate credit risks. Without adequate data, AI systems can fail to generalize well across scenarios, or can make poor predictions and provide unreliable outputs (World Bank, 2023).

AI has grown due to the surge in digital information. The digital information is becoming more available, which has propelled AI development. Data is created every second as people interact online, use their social media, shop on the internet, use mobile devices, wearables, sensors, and the Internet of Things technologies. Governments and organizations gather massive data that can be analysed and then used for machine learning. This wealth of information has facilitated the creation of more complex AI systems that can solve problems in various industries. The wide gathering and usage of personal data, however, also raises serious concerns about privacy, consent and ethical data handling.

Continuous learning is also another key factor of AI data dependency. A lot of modern AI systems are engineered to learn and grow as they get new data inputs. Machine learning models can evolve as per the varying conditions, user behaviors, and the environment, unlike static systems. This feature enables AI apps to stay current and precise in the ever-changing landscape. For instance, streaming services and ecommerce platforms continuously analyze user interactions to make more accurate recommendations in the future. Likewise, cybersecurity systems can be trained to learn new information about threats and enhance their detection and response capabilities to cyberattacks. Even though continual learning improves performance, it also poses governance issues, as the behaviour of AI systems can evolve over time, making it hard to predict and monitor (UNCTAD, 2024).

Data quality and accuracy are another important factor in the success of AI. If information fed into AI systems is inaccurate, so will be their results. Any outputs may be flawed or misleading if data is incomplete, outdated, inaccurate, or manipulated. In simple terms, this can be expressed as “garbage in, garbage out” (GI-GO), emphasizing the tight correlation between the quality of the data and the quality of the AI output. Poor-quality data can have serious repercussions in certain industries, such as healthcare, finance, and public administration, where it can result in misdiagnoses, miscalculations, and mistakes in public policy. Therefore, safeguarding the accuracy, integrity, and reliability of data is a key pillar of responsible AI governance (Kshetri, 2022).

The variety of training data also is significant. An AI system developed on a small or unrepresentative sample of users could not necessarily work well in other groups or settings. Facial recognition algorithms, for example, have been shown to have worse accuracy for specific ethnic groups when they are not sufficiently diverse. These issues are also seen in the recruitment algorithms, predictive policing and credit assessment systems. It's therefore crucial that you have datasets which are inclusive of various population groups to minimise bias and make certain that A.I.-based decision-making processes are fair.

Another important component of AI data dependency is behavioral profiling. AI systems have the ability to identify patterns in human behavior and predict future actions, preferences, and decisions. By using various data sources such as browsing history, purchase records, social media data, location data, and online interactions, organisations can create comprehensive profiles of users and user groups. Targeted ads, personal services, risk assessment, and predictive analysis are just some of the uses to which these profiles are put. The use of behavioral profiling could help to improve user experiences and potential service delivery, but also has privacy, autonomy and manipulative effects on individual behaviours concerns (World Economic Forum, 2024).

The growing use of behavioral data has a major impact on personal privacy. A lot of people don't realize how much of what they do is watched, studied, and used in predictive models. AI systems can make assumptions about sensitive data, preferences, beliefs, and future intentions, based on seemingly unrelated data points. These capabilities raise issues of informed consent, transparency, and the possibility of misuses of personal information. In some instances, a person might be subject to automated decision making or targeted interventions, without a full understanding of how decisions were made.

Another difficulty is the concentration of data into a number of relatively small technology companies and government institutions. Large organizations may have a lot of benefits in terms of AI development due to having access to massive sets of data and computational power to allow for more sophisticated machine learning. The centralization of such data can lead to power dynamics in digital ecosystems, which would raise questions of dominance, competition, data monopolies, and equalizing access to the technology. Increasingly, policymakers understand that data governance regimes should not just be about privacy but other issues of economic power and digital equity (OECD, 2023).

Data dependency is one of the most important cybersecurity risks of AI. The more data that's collected and stored within an organisation, the more appealing it is to cybercriminals. AI training sets can contain sensitive information that may lead to data breaches and impact the integrity of machine learning systems. Furthermore, training datasets can be compromised by data poisoning, which involves tampering with data to alter the outcomes of the AI model. Additionally, there are methods used by malicious individuals to manipulate the training data, such as data poisoning, which involves providing the AI with inaccurate or misleading information. These attacks can lead to loss of system reliability and significant security issues.

One of the other factors to consider is the difficulty of data governance on a global scale. Datasets frequently come from different countries and jurisdictions to feed into AI systems. The differences in privacy laws, data localization requirements and regulations on cross-border transfers may make it difficult to gather and leverage data for the development of AI. Compliance with a mix of regulatory standards is a complex challenge for global organizations, as is compliance with the complexities of legal requirements. These are the times when interoperability of regulations and international cooperation is increasingly important for the governance of AI and data-driven technologies (UNCTAD, 2024).

The reliance on data is a key governance issue for AI. Any imperfections, predispositions, misrepresentations, and inaccuracies in the data will likely be present in AI-generated content and can be exacerbated by automated systems. Bias in data can result in bias in outcome, incorrect information can result in incorrect predictions and incomplete data can result in incomplete decision making. The implications of poor data quality are growing more profound and widespread as AI systems are woven into the fabric of our society in critical areas.

Therefore, robust data governance practices are essential for effective AI governance, focusing on data quality, accuracy, transparency, accountability, security and fairness. It is crucial for organizations to have robust data management structures, regularly audit training data sets, and ensure that data collection practices meet legal and ethical requirements. Additionally, policymakers need to create a regulatory framework that is designed to tackle the distinctive challenges of AI data dependency, while fostering innovation and safeguarding the fundamental human rights.

Finally, AI and data are one and the same in today's digital world. The quality, quantity, diversity, and governance of the data that drives artificial intelligence (AI) systems have a direct impact on the performance, reliability, and societal impact of these systems. Data-driven AI presents immense potential for innovation and economic growth, but also poses critical privacy, bias, security, accountability, and fairness challenges. This interdependence is crucial for creating robust governance structures that can harness the full potential of AI technology while mitigating potential risks.

15.2 Algorithmic Bias

Algorithmic bias is a crucial ethical and governance issue in the context of AI and data-driven systems. When AI systems produce unfair, discriminatory or systematically skewed outcomes that affect certain individuals or groups. While AI technologies are typically regarded as neutral and objective due to their mathematical models and computational procedures, they learn from data created by human societies. This can lead to biases, inequalities, and historical patterns being reproduced and magnified by AI systems. The challenges of algorithmic bias have risen to the forefront of debates on algorithmic ethics and responsible governance of AI in the context of its embedding into critical aspects of social, economic, and political life (Barocas, Hardt, & Narayanan, 2019).

Algorithmic bias is a problem because AI systems don't make judgements about what is fair and equitable on their own. Rather, they discover patterns in past data and base their predictions/recommendations on those patterns. Distortions, omissions, stereotypes, and/or discriminatory outcomes in the data used for training can also be learned by the AI system and replicated. Sometimes, algorithmic bias can arise even if developers are not aiming to make discriminatory algorithms. This is because machine learning models are designed to be accurate and predictive, and not necessarily take into consideration ethical, social, and legal issues (EU, 2023).

Often, one of the primary causes of algorithmic bias is historical data bias. Historical data is often the product of social inequalities, discriminatory practices and institutional biases. AI systems that learn from this information can perpetuate the status quo by repeating past patterns, which may result in continued inequalities. If the past hiring practices have discriminated against groups of people, for instance, an AI-driven recruitment process that has been trained on past hiring data could still perpetuate this bias even if the organisation is looking to hire diverse candidates and promote equal opportunities. Thus, AI systems can inadvertently continue to perpetuate past discrimination instead of solving it (Kshetri, 2022).

Another very strong reason for algorithmic bias is sampling bias. A sampling bias is a bias that happens when the training data is not representative of the population the AI system is supposed to be used for. The system may not perform as well for certain population groups if they are not overrepresented or included in the training data set. The issue has been noted in facial recognition software that has been developed mainly on photos of certain demographic subsets and that fails to recognize people from less-popular populations. The difference can create a significant issue if the technology is applied in a different context, like law enforcement, border security or public surveillance.

Discriminatory outcomes can be caused by measurement bias. This type of bias is created when the variables used to measure or represent real world phenomena are inaccurate, incomplete or systematically distorted. An AI system to evaluate employee performance, for instance, might use productivity measures that miss crucial factors of job performance. Likewise, predictive policing systems can rely on arrest data to detect crimes, though it is possible that arrest rates may reflect over-policing in some communities historically. In these cases, the information used to train AI systems might not reflect reality that is being modeled (World Bank, 2023).

Another critical aspect of model development that can lead to algorithmic bias is human decision bias. There are many choices that developers and data scientists have to make along the way in developing AI, such as picking training sets, objecting the objectives, picking variables, and picking test criteria. The decisions may be based on conscious and/or unconscious assumptions, views and priorities. Consequently, human biases may be ingrained in AI systems, even if they seem to be based on technical methods. This underscores that algorithmic bias isn't just a technical problem; it also demonstrates how human decisions and institutional practices are embedded in algorithms.

Algorithmic bias can have serious and widespread effects. Employment and recruitment is one of the most popular concerns. AI-powered tools are becoming more common in job screening, candidate evaluation, and hiring processes.AI tools are becoming a staple in the job screening, assessment, and hiring processes. These systems can provide increases in efficiency, but can be unfair towards individuals according to their gender, race, age, socioeconomic status, or other characteristics if the algorithms are biased. These results may contribute to perpetuate inequalities in the workplace and reduce the employment opportunities for some groups.

Algorithmic bias can also manifest itself in access to financial services. AI systems are employed in finance to assess creditworthiness, evaluate risk, and determine loan eligibility and financial product suitability by banks, insurance companies and financial institutions. When such systems are based on skewed information or incorrect assumptions, they can deny credit or attract discriminatory terms to some people or groups. This can contribute to financial exclusion and perpetuate existing economic disparities (UNCTAD, 2024).

The criminal justice and law enforcement industries have also seen considerable impacts related to concerns about algorithmic bias. The use of predictive policing systems and risk assessment algorithms is becoming common in making decisions about crime prevention, sentencing, parole and resource allocation. It has been proved, however, that biased training data can lead to systems that disproportionately discriminate against certain communities or demographic groups. Such results present significant questions of fairness, due process and equal treatment.

Another consequence of algorithmic bias is in healthcare. AI systems are widely employed to aid in the diagnosis, treatment planning, resource allocation, and patient risk assessment. Healthcare algorithms can yield less accurate results for specific population groups if they are not well-represented in their training sets. This can lead to inequities in access to health care, health outcomes and quality of care. The implications of healthcare decisions are immense, making it crucial that researchers and policymakers tackle the issue of bias in medical AI systems.

Barocas, Hardt, and Narayanan (2019) remind us that algorithmic bias cannot simply be seen as a problem with algorithms but as a symptom of wider inequalities in society that are ingrained in data systems. AI technologies function in a social context which is influenced by historical, economic, political and cultural conditions. Addressing algorithmic bias thus extends beyond the technical adjustments, and involves as much as social structures that impact unequal outcomes.

The first step in combating algorithmic bias is to create and apply diverse and representative data sets. Organizations should make sure that the training data adequately represents the populations and contexts in which AI systems are deployed. A variety of data can help mitigate the risk of systematic exclusion or disadvantage of certain groups. But diversity in data is not enough. Ongoing monitoring and evaluation is essential to detect new biases and measure AI system performance by demographic groups.

Another key factor of bias mitigation is transparency in model design. The organizations should clearly document AI system development, training, and evaluation. Greater transparency to better inform regulators, researchers and others affected by the decisions and to determine potential sources of bias. Other techniques for making AI systems explainable can help to improve transparency further by allowing for more interpretable and accessible processes in complex algorithmic systems.

It's also important for AI systems to undergo regular audits to detect and rectify discriminatory outcomes. Algorithmic audits refer to the systematic evaluation of the performance, fairness, accuracy, and legal and ethical compliance of systems. There are times when independent audits are useful for their oversight and to identify problems that may not be seen in the early phases of development. Continual auditing becomes even more critical as AI systems continually learn and evolve to ensure accountability and trust.

Ethical governance frameworks are an important aspect of the solution to algorithmic bias. These frameworks tend to focus on values like fairness, transparency, accountability, human oversight, non-discrimination and respect for human rights. Numerous governments, international institutions and tech firms have devised guidelines for AI ethics, aiming to foster responsible innovation. Ethical principles may not be sufficient in themselves to ensure there is no bias, but they can form a valuable starting point for creating governance mechanisms that will balance technological progress and the values of societies (World Economic Forum, 2024).

This is also due to the increasing awareness of algorithmic bias, which have been the subject of calls for tighter regulations. Globally, governments are considering legislation that mandates fairness testing, transparency reporting, assessments of impacts and accountability for dangerous AI systems. These laws are designed to prevent discriminatory outcomes from occurring and require proactive measures by organisations deploying AI technologies to identify and mitigate such outcomes.

To wrap up, algorithmic bias is a single of the most substantial obstacles to current AI governance. Bias can manifest in historical data, in small sample sizes, due to measurement error, and because of decision-making processes by humans and their interactions, leading to discriminatory outcomes in the critical areas that include employment, finance, healthcare, law enforcement and others. This problem transcends technology and is symptomatic of overall social inequities that exist in data systems. An integrated solution to the issue of algorithmic bias involves using a variety of data sources, designing open models, regularly evaluating the models, creating ethical governance models, and having strong regulatory bodies for effective oversight. Responsible digital governance will continue to be a key priority of fairness and equity in AI systems that are playing an increasingly pivotal role in decision-making across individuals and societies.

15.3 Facial Recognition Technologies

One of the fastest-growing applications of the artificial intelligence and biometric identification systems is facial recognition technology (FRT). Its features include the use of advanced AI algorithms, machine learning models, and computer vision technologies to identify, verify or authenticate people based on unique facial features. Facial recognition systems can capture specific facial characteristics like the distance between the eyes, shape of the nose, jaw structure, and many other biometric traits to build a digital template and match it against a database of templates to identify a person. With the improvement of the capabilities of AI, facial recognition technology has become more accurate, scalable, and widely available, and has now been widely used in both the public and private sectors (Russell & Norvig, 2021).

As the use of facial recognition becomes ubiquitous, it mirrors the larger digital transformation and data-driven governance. Facial recognition technology has become a key component in the security and identification systems of governments, law enforcement agencies, companies, airports, financial institutions and technology firms, with the aim of bolstering security, streamlining operational processes and automating identity check procedures. These technologies have many advantages, but they also come with serious privacy, civil liberty, accountability and human rights issues. For this reason, facial recognition is one of the most controversial uses of AI in today's society (OECD, 2023).

This is one of the most prevalent uses of facial recognition technology, especially in border control and immigration. In many countries, there are biometric identification systems at airports and border crossings to identify travelers, improve security screening and facilitate the processing of passengers. Automated passport controls can match the live face image with the passport photograph in seconds, which can lower turnaround time and enhance efficiency. This is a trend that sees these systems being built into smart border management programmes that integrate fingerprint, facial and iris recognition.

Facial recognition is also widely used by the public security and law enforcement sector. Police departments and security agencies can use facial recognition technology to help identify suspects, find missing persons, and track people in public areas and aid criminal investigations. AI facial recognition technology can process video data instantly and match what it sees with criminal databases or watchlists. Advocates say these systems help to increase public safety and detection of criminals. But critics caution that widespread surveillance could be counterproductive in relation to privacy rights, and present opportunities for abuse unless adequate safeguards are put in place (United Nations, 2023).

Facial recognition capability has been getting more widespread in consumer technology and digital services. Facial recognition is a technique used by smartphone manufacturers to provide authentication of the smartphone, allowing a person to unlock the device and use applications without using a password. Digital banking uses facial recognition to verify identity when performing financial transactions or during customer onboarding. Facial recognition can be used in social media to tag photos, categorize content, or enhance user experience. The commercial uses indicate the ease and efficiency that comes with biometric authentication systems.

Facial recognition has also been implemented in the retail and marketing sector, where customer analysis and business intelligence are being used. Facial recognition systems are employed by some retailers to gain insight into the demographics of their shoppers, track their shopping patterns, understand how they interact with products, and tailor marketing strategies to individual consumers. The data can be leveraged to design better store layouts, create more personalized customer journeys, and make informed decisions about business operations. Such applications have commercial benefits, but they also present issues related to transparency, informed consent and the ethical utilization of biometric data (World Economic Forum, 2024).

However, facial recognition technology is fraught with privacy concerns. The first worry is that there might be an infringement on individual privacy rights as they are monitored and surveilled at all times. Unlike other data collection methods, facial recognition can be used to identify people without their awareness or cooperation. They can be followed without their explicit consent in public places, public transportation, workplace, shopping centers, and online. This technology can profoundly transform the way people interact with public areas, posing issues of anonymity, autonomy, and personal liberty (UNCTAD, 2024).

Also, there is a potential for misidentification. In recent years, the technology has come a long way, but facial recognition systems are not perfect. Poor image quality, lighting, face occlusion, age or data set limitations can cause errors. There have been studies that have revealed that some facial recognition systems have different success rates for different demographics, especially when the data used for training is not diverse. Misidentification can have serious consequences when facial recognition is used in law enforcement or judicial contexts, potentially leading to wrongful accusations, mistaken arrests, or unjust legal outcomes. The risks emphasize the need to ensure the human presence and enforce strict accuracy requirements in high-stakes applications (Buolamwini & Gebru, 2018).

One of the most prominent controversies surrounding facial recognition technology is mass surveillance. Governments and other bodies can monitor individuals in large populations in ways that were previously impossible because they could not identify and track individuals in a population. The big data surveillance system provides the ability to gather huge amounts of biometric data and create extensive files of how people move, act and interact. Governments might argue that this is for national security, public safety, or crime prevention reasons, but critics say that an overzealous surveillance system can limit democratic freedoms, deter legitimate public expression, and chill free speech or assembly. These are especially important in jurisdictions where there is a low level of independent oversight or where there are not enough laws to stop misuse.

The governance of facial recognition technologies is made more difficult by consent and transparency issues. People often do not realize that their facial information is being captured, stored or analyzed without their knowledge. Unlike passwords or identification cards, facial features are more difficult to alter if they were breached. Therefore, improper collection and/or usage of biometric data can pose lasting privacy and security issues. While face recognition is deployed passively or continuously in certain contexts, many data protection regulations call for informed consent of the data subject for the collection of sensitive personal data, but implementing such consent mechanisms can be challenging (OECD, 2023).

Another key governance issue is data security. Facial recognition systems are based on the storage and interpretation of highly sensitive bio-metric data. Affected individuals could be the victims of identity theft, fraud, unauthorized surveillance, or other forms of harm if these databases are compromised, stolen, or improperly accessed. Unlike account numbers or passwords, facial details cannot be changed once they've been compromised. Therefore, organisations using facial recognition technologies need to put in place strong cyber security and data protection security measures.

It's not only privacy and security that are an issue with facial recognition. The large-scale implementation of biometric identification technologies poses a wider justice, proportionality, human dignity and societal trust issue. There are some scholars who say that it could be a change in social behaviour, that people feel like they are being watched all the time. This can impact people's independence and reduce trust in public bodies. The debate on ethics is increasingly on whether there should be any uses of facial recognition restricted or banned entirely, especially in mass surveillance (Floridi et al., 2018) or high-risk decision-making situations.

Facial recognition technology is subject to different regulations in different jurisdictions. In some countries, the use of biometric surveillance systems is strictly limited, in others, less so. Some cities and local governments have banned or put moratoriums on the use of facial recognition by law enforcement agencies, because of privacy and civil rights concerns. However, other nations are still rolling out facial recognition systems in their digital governance and security frameworks. The variety of regulatory solutions is a sign of the constant trade-offs between technical innovation and safeguarding of fundamental rights and freedoms.

It is evident that international organisations and policy makers are taking note of the need for a comprehensive governance framework for facial recognition technologies. These frameworks will typically include the concepts of necessity, proportionality, accountability, human oversight, non-discrimination, respect for privacy rights, and transparency. Effective governance must not only be legally protected, but also must be backed by independent oversight mechanisms, regular audits of the system, impact assessments and public accountability measures.

To sum up, facial recognition is one of the most effective and debatable AI implementations in today's digital age. Its increasing applications in border control, policing and consumer electronics, as well as commercial applications, prove the great practical value and potential of this new technology. But questions about privacy abuses, misidentification, mass surveillance, consent, data security and civil liberties have continued to spark heated public discussion. In a rapidly evolving landscape of digital societies, establishing comprehensive and effective governance systems will be crucial to prevent the development of facial recognition technologies from impacting negatively on human rights, privacy, and public trust.

15.4 Predictive Analytics

Predictive analytics involves the use of statistical models and AI algorithms to predict future outcomes based on historical and real-time data.

Predictive Analytics

Common Applications:

•Credit scoring

•Fraud detection

•Customer behavior prediction

•Healthcare risk assessment

•Criminal justice forecasting

Risks:

1. Over-Reliance on Predictions

Organizations could be overly dependent on algorithmic results.

2. Data Misinterpretation

Misaligned information may result in misaligned forecasts.

3. Reinforcement of Inequality

Bias in the past can be reinforced in forecasts.

4. Lack of Transparency

There are many predictive models that aren't easy to understand.

Predictive analytics brings up key ethical dilemmas regarding: fairness, accountability and the boundaries of algorithmic decision-making.

Manipulation of information and deepfakes. Manipulation of information and deepfakes (15.5).

Deepfakes are synthetic media created through AI technology, capable of reshaping and/or falsifying video, audio and images.

Deepfake Technology

Applications:

Production of entertainment and media products

•Digital marketing

•Content personalization

Risks:

1. Misinformation and Disinformation

The deepfakes may be employed to share misinformation.

2. Political Manipulation

Fake videos can sway elections and opinion.

3. Reputation Damage

Content may be misleadingly presented about individuals.

4. Cybercrime and Fraud

Deepfakes can be utilized for impersonation frauds and monetary scams.

With the advancement of deepfake technology, information integrity has become a significant problem in the digital era.

15.6 Future Threat Landscape

The rapidly developing threat landscape of Artificial Intelligence (AI) and Big Data mixes with growing societal, economic, political and security-related risks while providing for fast advancing technology. As businesses, governments, and individuals increasingly rely on AI-powered systems and data-driven technologies, the risks and exposures associated with these tools continue to grow and pose a threat to privacy, security, human rights, and world peace. AI presents immense opportunities and promises of innovation and efficiency, but it also poses new challenges that demand proactive governance, regulatory oversight, and ethical protections.

Emerging Threats

1. Autonomous Decision-Making Systems

An important challenge for the future is AI systems making critical decisions with very limited or no human involvement. In recent years, these systems have been increasingly employed in the medical, financial, legal, defense, transport and public administration sectors. Although automation can assist in boosting efficiency and minimizing human error, autonomous systems are susceptible to making decisions influenced by inaccurate information, faulty algorithms, or biased data. These kinds of mistakes may lead to discriminatory results, misidentification, monetary loss, or even life-threatening encounters. Moreover, establishing responsibility in cases of incorrect decisions by an AI system is still a huge legal and ethical problem.

2. Mass Surveillance Expansion

AI technologies are now being incorporated into surveillance systems more and more by the governments and organizations. Facial recognition, biometric monitoring, predictive analytics and behavioral tracking technologies can gather and analyze a vast amount of personal data in real time. The technologies could improve national security and public safety, but pose serious concerns about privacy, civil liberties, and governmental overreach. If not properly controlled, AI-powered surveillance tools can play a role in the creation of highly intrusive environments of surveillance that violate democratic freedoms and individual rights.

3. AI-Powered Cyberattacks

AI is changing the way cybersecurity is done both offensively and defensively. AI can be used by cybercriminals to carry out automated attacks, exploit system weaknesses, create sophisticated malware and launch extremely targeted phishing attacks. Cyberattacks powered by AI are likely to occur at an unprecedented speed and scale, making their detection and mitigation more challenging. Deepfake technologies can also be employed to impersonate people, alter communications and conduct social engineering attacks. The more that organizations can do with AI, the more they need to do to protect themselves against AI threats as they get more powerful.

4. Data Weaponization

Data is now strategically valuable, which can be used for political, economic and military purposes. Massive databases can be exploited by government, businesses, and bad guys to sway people, to cheat in elections, to spy on commerce, or to give them the edge. Social media analysis, behavioural profiling and targeted misinformation campaigns can help to manipulate public opinion and alter societal behaviour. The weaponization of data is a grave danger for democratic institutions, national security and public trust.

5. Synthetic Reality Systems

The evolution of generative AI is facilitating the production of extremely convincing synthetic content such as deepfake videos and audio, virtual environments, and digitally synthesized identities. These synthetic reality systems could have a tendency to make it harder to tell real facts from fake ones. Misinformation, reputation attacks, political manipulation, and the loss of confidence in digital media are some of the many ways in which deepfakes can be exploited. With the rise in the number of fake contents, social societies can encounter more difficulties in verifying information and build trust in digital communication.

7. Loss of Human Oversight

With the growing adoption of AI-driven automation, there is a heightened risk of diminishing meaningful human supervision and accountability. Relying too heavily on automated systems could cause people to believe the results of algorithms without verifying or critically assessing them. This is known as automation bias and can lead to suboptimal decision making and less human responsibility. The critical decision-making process cannot be carried out without man-in-the-middle and man-in-the-loop to make it transparent, accountable, and ethical.

8. AI-Driven Disinformation Campaigns

So, future AI systems could enable the scale and effectiveness of disinformation campaigns significantly. Generative AI can quickly generate realistic text, images, video, and social media posts that are meant to sway public opinion or disseminate false narratives. These campaigns can be employed to sway elections, generate social discontent, weaken institutions and/or disrupt international relationships. Generating persuasive content at scale with AI poses a major challenge to governments, media outlets and fact-checking institutions.

9. Critical Infrastructure Vulnerabilities

AI and Big Data technologies are now being adopted in critical infrastructure systems in many industries, such as energy, healthcare, transportation, telecommunications and financial services. In addition to making the operations more efficient, these technologies also create new attack points for cybercriminals and malicious actors. Losing control of AI-infused infrastructure could impact critical infrastructure, economic activity and public safety on a massive scale.

Concentrating Technological Power.9. Centralisation of Technological Power.

Advanced AI technologies can demand a considerable amount of computational power, large amounts of data and financial resources. This could lead to the creation of disproportionate control of AI power by a handful of large tech companies and strong governments. This technological power may lead to economic disparities, decrease competition and lead to geopolitical tensions. Having fair access to AI technologies will be one of the critical policy issues of the next decade.

10. Existential and Long-Term AI Risks

There are fears about the long-term implications of superintelligent AI systems among some AI researchers and policymakers. Although these risks are largely speculative, there are fears that future AI systems might function in ways that are hard to anticipate, manage and conform to human values. As AI's capabilities continue to grow, discussions on topics like AI alignment, AI safety research, and global governance have become more significant than ever.

Beyond cyber security, the future threat landscape of AI and Big Data is a far more complex one that includes privacy, governance, social stability, economic security and human rights issues. As new challenges arise, including technologies like autonomous decision-making systems, mass surveillance, AI-driven cyberattacks, data weaponization, synthetic reality technologies and the reduction of human oversight, well-designed governance mechanisms are essential. Governments, international organizations, technology companies and civil society need to collaborate to develop adequate regulatory policy, ethical policy guidelines, transparency policies and cooperation frameworks. The benefits of AI and Big Data can only be achieved through responsible innovation and proactive risk management by society.

Artificial Intelligence and Big Data are transforming the global digital landscape, offering unprecedented opportunities for innovation, efficiency, and economic growth. However, they also introduce complex risks related to bias, privacy, surveillance, misinformation, and systemic inequality.

As demonstrated in this chapter, the dependency of AI on data makes it both powerful and vulnerable. Issues such as algorithmic bias, facial recognition misuse, predictive inaccuracies, and deepfake manipulation highlight the urgent need for robust governance frameworks.

Future digital systems must prioritize transparency, accountability, fairness, and human oversight to ensure that AI technologies serve society ethically and responsibly.

The next chapter will examine global policy responses and regulatory approaches to managing AI and emerging digital risks in an increasingly interconnected world.