Defining Your AI Product Objective

Optimize for long-term quality and be aware of all the unintended negative outcomes, let me explain how.

The right objective

Defining the right objective for your algorithm is the most essential and potentially the most difficult job you’ll ever have as a product leader or manager, or owner. You might achieve the exact opposite result if you don’t carefully evaluate what you want to achieve. You must begin thinking quickly and slowly about your user outcomes in order to choose the right objective.

Start with your instincts about what the goal should be, but then take a more deliberate, methodical approach to it, examining all the details. Always use data to back up your hypothesis, and be careful not to mix it up with causation and correlation. For example, you could optimize the algorithm for the number of times job seekers click on jobs. Isn’t this straightforward? It’s valuable to them if they’re clicking on it. That, however, is incorrect. Clicks may be a poor indicator of relevance. Let me explain.

Ideally, job seekers will engage with only those jobs in which they can succeed, rather than all jobs. Not only for job seekers, but also for recruiters, this will result in a more positive outcome. I recommend first answering the following two questions in order to carefully choose the right goal.

To begin with, what are your long-term goals?

What would the end result look like if your algorithm is successful?

I’ll give you a helpful hint: the end result is usually a mix of outcomes because a single outcome is usually too narrow.

The same can be said for your goals. Strong AI goals are typically a mix of different goals. Let’s take Amazon for example. Amazon’s website should not be optimized primarily for clicks. Even purchases may not be sufficient.

Should they optimize for non-returnable purchases?

Or should they focus on optimizing for Amazon Prime memberships?

As you can see, deciding on the right goal is a difficult task. However, the best approach is to always consider long-term applications and to continue experimenting in…