Designing algorithm checking procedures

In order to address the proliferation of machine learning-algorithms, new regulatory frameworks and policy initiatives increasingly checks of such algorithms. Common checking procedures cannot address all the challenges of algorithm checks outright. For instance, typical one-off product tests cannot address that algorithms often change frequently.

We use the market design toolbox to address such challenges. We view algorithm checks as information production processes, which can be optimized within – potentially conflicting – constraints.

• Turing Markets
  Competition to develop algorithm checking procedures for a close-to-real life black-box algorithm, with most promissing procedures to be tested in lab in the field-experiment.
  Funded by Baden-Württemberg Foundation.
• Judging algorithmic behavior
  Elicitation of judgements on the behavior of machine learning algorithms, at scale, under cost constraints, capturing full heterogeneity and potential ambiguity in judgements.

Designing markets for data

Numerous policy and industry initiatives try to start marketplaces or even multi-service platforms for data. In theory, data provision has zero marginal costs for sellers. In practice, however, the preparation of existing data and collection of new data is costly and demand for data is difficult to assess. Furthermore, data often only generates utility for buyers in combination with other data and the utility of data can be very heterogeneous among buyers.

We are interested in developing building blocks of markets for data, many of which could also be incorporated in algorithm checking procedures.

• Combinatorial data valuation
  Developing economic model for combinatorial data valuation and run Kaggle competition to gather data in order to calibrate model to common data valuation scenarios.
  Funded by German Federal Ministry of Education and Research as work package of research project Incentives and Economics of Data Sharing (IEDS).

Detecting and preventing misbehavior in digital markets

As markets become more and more digitized, both old and new types of misbehavior of market participants arise. Collusion, fraud, discrimination or other forms of misbehavior often take new forms, which are sometimes easier and sometimes more difficult to detect and prevent.

We are interested in uncovering misbehavior in digitized markets and developing changes in the respective's market design to alleviate them.

• Preventing algorithmic collusion
  Test of different market designs to prevent algorithmic collusion using concepts from adversial machine learning and differential privacy.
• Preventing discrimination on online rental marketplaces
  Redesign of information provision in tenant screening processes to alleviate discrimination on online rental marketplaces, which match the majority of rental housing supply and demand.

Enabling digital health innovations

Digitizing health care has great potential. For instance, the digitization of health records could develop into a particular form of digital platform enabling new health treatments such as precision medicine. The SARS-CoV-2 pandemic is likely to serve as a catalyzer in this direction. However, the mixed reception and success of contact tracing apps during the pandemic also provides a cautionary tale. Raising the full potential of such innovations is no easy feat.

We are interested in developing methods and tools to encourage participation in such innovations.

• Fostering participation in digital contact tracing apps
  Literature review on how to foster participation in digital contact tracing apps and other digital public health interventions: Research paper, policy brief, podcast, exemplary press coverage.
• Improving privacy literacy
  Test of treatments to increase privacy literacy and assessment of conversion into actual participation in digital health innovations.

Adressing economic challenges of free digital products and services

Many essential digital products and services are provided to consumers essentially for free. Yet, consumers indirectly pay for such services, for instance, by paying attention to ads or by providing personal data. This causes a number of issues, such as distortions in the measurement of welfare or challenges in the definition of markets relevant to competition policy.

We are interested in the empirical measurement of the consumer surplus of free digital products and services, which can provide valuable information to deal with such issues.

• Determining free subsitutes and complements
  Elicitation of substitutability and complementarity of free digitial products and services, at scale, capturing heterogeneity and changes over time.
• Assessing value of social connectedness
  Assessment of value of social ties provided by online social networks.

• Targeted sampling under budget constraints
  Method to determine the value of sample units for estimating a parameter of interest and to target particularly valuable sample units.
• Measurement order bias
  Method to test for and to address non-randomness in taking measurements from sample units.