The Sketch2Code website created by Microsoft generates HTML code from sketches, such as for the design of an online shop or the different options on a payment page. The software also accepts smartphone camera photos that can be uploaded directly. The technology is based on Microsoft's Cognitive Services , a collection of interfaces for image analysis or speech recognition. They were specially trained to recognize HTML elements in handmade drawings for the new task. An additional text recognition system converts existing handwritten entries in the sketches into matching annotations.

Microsoft apparently used several million drawings to train Sketch2Code. The system very reliably recognizes elements such as images or lists. Recognition accuracy is continuously improving based on the sketches users upload.

Sketch2Code can also be used for your own sketch analysis; a corresponding repository containing the software's code is available on GitHub . Microsoft has published further details in a blog posting and in its AI Lab .