Eight Microsoft interns have developed a new language learning tool that uses the smartphone camera to help adults improve their English literacy by learning the words for the things around them. The app, Read My World, lets you take a picture with your phone to learn from a library of more than 1,500 words. The photo can be of a real-world object or text found in a document, Microsoft says.

 

The app is meant to either supplement formal classroom training or offer a way to learn some words for those who didn’t have the time or funds to participate in a language learning class.

 

Instead of lessons, users are encouraged to snap photos of the things they encounter in their everyday lives.

 

“Originally, we were planning more of a lesson plan-style approach, but through our research and discovery, we realized a Swiss army knife might be more useful,” said Nicole Joyal, a software developer intern who worked on the project. “We wound up building a tool that can help you throughout your day-to-day rather than something that teaches,” she said.

 

Read My World uses a combination of Microsoft Cognitive Services and Computer Vision APIs to identify the objects in photos. It will then show the word’s spelling and speak the phonetic pronunciation of the identified vocabulary words. The photos corresponding to the identified words can also be saved to a personal dictionary in the app for later reference.

 

Finally, the app encourages users to practice their newly discovered words by way of three built-in vocabulary games.

 

Key features include:

  • Take a photo to identify an object from a library of over 1500 vocabulary words
  • Take a photo to identify vocab words in documents or other written mediums
  • See the spelling and hear the phonetic pronunciation of identified vocab words
  • Save photos with corresponding identified word to a personal dictionary for later reference  Practice saved words with any of three vocabulary games.