Cognitive Services: The Big Picture

Introduction:

This blog will discuss the basics of cognitive services and the complex machine learning processes that fuel them. This blog will also focus on how these services allow you to make your applications intelligent.

Cognitive services are pre-trained machine learning models that are exposed as Restful API’s, to be consumed directly in any application.

Machine learning Workflow:

To be able to understand the concept of cognitive services, we first need to understand the basis of machine learning workflow. The high-level workflow of machine learning has these steps:

  • Collecting data
  • Pre-processing data
  • Training machine learning model
  • Testing the model
  • Deploying the model as a web service

These steps are not as simple and straightforward as they seem. Collecting data and preprocessing your data to train a neural network or machine learning model requires a lot of effort, time and cost.

Cognitive services landscape:

To make things easier, big tech giants such as Google, Microsoft and IBM do the hard part for you. They gather data, preprocess it, train machine learning models and provide the REST endpoints that you can directly use in your applications without having to go through the whole machine learning life cycle. All you need to do is subscribe to the service and use it in your applications.

There are many vendors out there that provide cognitive services:

  • IBM through IBM Watson
  • Microsoft through Azure
  • Google through Google Cloud Platform
  • Amazon through Amazon web services

IBM became the pioneer of cognitive services by offering them in September 2015 followed by Microsoft and Google in March 2016 and then by Amazon in November 2016. Currently, Microsoft provides the most number of cognitive services (more than 20).

Understanding Cognitive services through an example:

One of the cognitive services provided by Microsoft under the Vision category is the computer vision service, that allows you to extract rich information from images.

To use this service, one must log on to portal.azure.com and create this computer vision service. This service will provide a REST endpoint and an authentication key that you can use to make POST calls with your images and receive information as a response. Simple? Isn’t it.?

At the back end however, Microsoft has trained a Convolutional Neural Network on millions of images. The trained machine learning model can be used as a service.

Categories of Microsoft Cognitive services:

Each category provides two types of services:

  • Regular services are pretrained model exposed as API’s.
  • Custom services use your own data for training rather than Microsoft’s data. This allows you to build a model that is specific to your use case.

1-    Vision:

  • This contains services that deal with images and videos
  1. Computer vision: This service extracts all sorts of information from an image.
  2. Face: It detects faces in an image and can extract information about them like age, color etc.
  3. Form recognizer: This API allows you to extract key-value pairs & other information from a form.
  4. Video Indexer: This API extracts text that is spoken in a video along with other information.

2-    Speech:

  • These services can be used to translate audio into text or for voice verification
  1. Speech Translation: This service provides you real time speech translation.
  2. Speech to Text: This API converts audio speech into readable text.
  3. Text to Speech: This service converts text into speech.

3-    Language:

  • This category helps you process natural language
  1. Text Analytics: This extracts user intent and important topics & named entities from the text.
  2. Translator text: This API allows you to translate text into different languages.
  3. QnA maker: This API converts your text into question answer pairs.
  4. Language Understanding: This service helps you incorporate language understanding in your apps.

4-    Decision:

  • Services in this category map information and data to make intelligent recommendations
  1. Content Moderator: This service detects offensive or unwanted content.
  2. Anomaly detector: This API helps you detect anomalies and outliers in your data.

5-    Search:

  • Services in this API are specialized Bing search API’s that help you with auto suggest, spell correction and more
  1. Auto suggest: This can suggest search terms.
  2. News search: To search news items.
  3. Web search: This API uses Bing to search the web.
  4. Entity search: This service identifies entity types like person, place etc.
  5. Image search: Helps you search images.
  6. Video search: Lets you search video.

Note: Microsoft is regularly updating the list of their cognitive services. New services are being added and some are removed or renamed. To keep up with these changes, you should regularly visit Azure AI for updates.

Each service is priced differently. Some are priced per amount of calls; other models allow fixed number of calls per month.  Microsoft provides a free pricing tier that allows only a limited number of API calls.

The purpose of writing this blog was to give you a brief overview of what cognitive services are, landscape of cognitive services and provide you with a detailed summary of the cognitive services catalogue offered by Microsoft. If you have any insights on cognitive services, feel free to leave a comment below.

If you have any question or queries, do not hesitate to reach out to us!