Monday at Microsoft Build 2018, Microsoft's annual developer conference, Microsoft showcased new technologies to help every developer be an AI developer, on Microsoft Azure, Microsoft 365 and across any platform. The company also previewed Project Brainwave and announced Project Kinect for Azure.
"The era of the intelligent cloud and intelligent edge is upon us," said Satya Nadella, CEO, Microsoft. "These advancements create incredible developer opportunity and also come with a responsibility to ensure the technology we build is trusted and benefits all."
Microsoft also today announced AI for Accessibility, a new $25 million, five-year program aimed at harnessing the power of AI to amplify human capabilities for more than 1 billion people around the world with disabilities. The program comprises grants, technology investments and expertise, and will also incorporate AI for Accessibility innovations into Microsoft Cloud services.
Advancements in the intelligent edge and intelligent cloud
- Microsoft is open sourcing the Azure IoT Edge Runtime, allowing its customers to modify, debug and have more transparency and control for edge applications.
- Custom Vision will now run on Azure IoT Edge, enabling devices such as drones and industrial equipment to take critical action quickly without requiring cloud connectivity. This is the first Azure Cognitive Service to support edge deployment, with Microsoft promising to bring more to Azure IoT Edge over the next several months.
- DJI, the world's biggest drone company, is partnering with Microsoft to create a new SDK for Windows 10 PCs, and it has also selected Azure as its preferred cloud provider to further its commercial drone and SaaS solutions. The SDK will bring full flight control and real-time data transfer capabilities to nearly 700M Windows 10 connected devices globally. As part of the commercial partnership, DJI and Microsoft will co-develop solutions leveraging Azure IoT Edge and Microsoft's AI services to enable new scenarios across agriculture, construction, public safety and more.
- Microsoft announced a joint effort with Qualcomm Technologies, Inc. to create a vision AI developer kit running Azure IoT Edge. This solution makes available the hardware and software required to develop camera-based IoT solutions. Developers can create solutions that use Azure Machine Learning services and take advantage of the hardware acceleration available via the Qualcomm Vision Intelligence Platform and Qualcomm AI Engine. The camera can also power Azure services, such as machine learning, stream analytics and cognitive services, that can be downloaded from the cloud to run locally on the edge.
Data and AI development
- Microsoft announced Project Kinect for Azure, a package of sensors, including the company's next-generation depth camera, with onboard compute designed for AI on the Edge. Building on Kinect's legacy that has lived on through HoloLens, Project Kinect for Azure empowers new scenarios for developers working with ambient intelligence. Combining Microsoft's Time of Flight sensor with additional sensors all in a small form factor, Project Kinect for Azure will leverage Azure AI to improve insights and operations. It can input fully articulated hand tracking and high-fidelity spatial mapping.
Project Kinect for Azure devices will go on sale next year and will let software developers write cloud applications that make use of sound, gestures or spatial understanding of the surrounding area. For example, a customer could place the devices in work sites to track things like spills or in a retail store as part of a cashless checkout experience, a similar idea to Amazon's Go store.
- A Speech Devices SDK announced today promise to deliver superior audio processing from multichannel sources for more accurate speech recognition, including noise cancellation, far-field voice and more. With this, developers can build a variety of voice-enabled scenarios like drive-thru ordering systems, in-car or in-home assistants, smart speakers, and other digital assistants.
- Azure Cosmos DB updates include new multimaster at global scale capabilities, designed to support both the cloud and the edge, along with the VNET general availability for increased security. With these new updates, Cosmos DB delivers even greater cost-effectiveness and global scale.
- A preview of Project Brainwave, an architecture for deep neural net processing, is now available on Azure and on the edge. Project Brainwave makes Azure the fastest cloud to run real-time AI and is now fully integrated with Azure Machine Learning. It also supports Intel FPGA hardware and ResNet50-based neural networks.
The first Brainwave service will speed up image recognition so it's almost instantaneous, according to Doug Burger, a distinguished engineer in Microsoft Research, who works on the company's chip development strategy for the cloud.
Microsoft buys the chips from Altera, a subsidiary of Intel, and adapts them for its own purposes using software, an ability that's unique to that type of chip.
One early client is electronics manufacturer Jabil Inc., which plans to use the service in factories where it has optical scanners that find possible product defects, including variations in tiny components.
- New Azure Cognitive Services updates include a unified Speech service with improved speech recognition and text-to-speech, which support customized voice models and translation. Along with Custom Vision, these updates make it easier for any developer to add intelligence to their applications.
- Microsoft is positionong Azure to be the place to develop conversational AI experiences integrated with any agent. New updates to Bot Framework and Cognitive Services will power the next generation of conversational bots enabling richer dialogs, and full personality and voice customization to match the company's brand identity.
- A preview of Azure Search with Cognitive Services integration. This new feature combines AI with indexing technologies so it's possible to quickly find information and insights, whether via text or images.
Multisense and multidevice experiences
- The new initiative we mentioned previously - Project Kinect for Azure - a package of sensors from Microsoft that contains Microsoft's time of flight depth camera, with onboard compute, in a small form factor - designed for AI on the Edge. Project Kinect for Azure brings together this hardware technology with Azure AI to empower developers with new scenarios for working with ambient intelligence.
One of the things that makes Project Kinect for Azure unique is the combination of our category-defining depth-sensor with Azure AI services that, together, will enable developers to make the intelligent edge more perceptive than ever before.
The technical breakthroughs in our time-of-flight (ToF) depth-sensor mean that intelligent edge devices can ascertain greater precision with less power consumption. There are additional benefits to the combination of depth-sensor data and AI. Doing deep learning on depth images can lead to smaller networks needed for the same quality outcome. This results in much cheaper-to-deploy AI algorithms and a more intelligent edge.
Earlier this year, Cyrus Bamji, an architect on our team, presented a paper to the International Solid-State Circuits Conference (ISSCC) on our latest depth sensor. This is the sensor that Satya described onstage at the Build conference and is also the sensor that will give the next version of HoloLens new capabilities. The technical characteristics of the new depth sensor include:
- Highest number of pixels (megapixel resolution 1024x1024)
- Highest Figure of Merit (highest modulation frequency and modulation contrast resulting in low power consumption with overall system power of 225-950mw)
- Automatic per pixel gain selection enabling large dynamic range allowing near and far objects to be captured cleanly
- Global shutter allowing for improved performance in sunlight
- Multiphase depth calculation method enables robust accuracy even in the presence of chip, laser and power supply variation.
- Low peak current operation even at high frequency lowers the cost of modules
- With Microsoft Remote Assist, Microsoft's customers can collaborate remotely with heads-up, hands-free video calling, image sharing, and mixed-reality annotations. Firstline Workers can share what they see with any expert on Microsoft Teams, while staying hands on to solve problems and complete tasks together.
- With Microsoft Layout, Microsoft's customers can design spaces in context with mixed reality. Import 3-D models to create room layouts in real-world scale, experience designs as high-quality holograms in physical space or in virtual reality, and share and edit with stakeholders in real time.
Microsoft is empowering developers to build for the new era of the intelligent edge, across Azure, Microsoft 365 and other platforms, using the languages and frameworks of their choice:
- With Azure Kubernetes Service (AKS), developers can simplify how they build and run container-based solutions without deep Kubernetes experience. Generally available in the coming weeks, AKS integrates with developer tools and workspaces, DevOps capabilities, networking, monitoring tools, and more in the Azure portal, so developers can write code, not stitch services together. In addition, Microsoft is now offering Kubernetes support for Azure IoT Edge devices.
- Visual Studio IntelliCode is a new capability that enhances everyday software development with the power of AI. IntelliCode provides intelligent suggestions to improve code quality and productivity and is available in preview today in Visual Studio.
- Visual Studio Live Share, now in preview, lets developers collaborate in real time with team members who can edit and debug directly from their existing tools like Visual Studio 2017 and VS Code. Developers can use Live Share with any language for any scenario, including serverless, cloud-native and IoT development.
- Microsoft announced a new partnership with GitHub that brings the power of Azure DevOps services to GitHub customers. Today, the company released the integration of Visual Studio App Center and GitHub, which provides GitHub developers building apps for iOS and Android devices to automate DevOps processes right from within the GitHub experience.
- Available today, the new Microsoft Azure Blockchain Workbench makes it easier to develop blockchain applications by stitching together an Azure-supported blockchain network with cloud services like Azure Active Directory, Key Vault and SQL Database, reducing proof-of-concept development time .
New developer on-ramps to Microsoft 365
Microsoft called on developers to embrace Microsoft 365 as a development platform. Microsoft 365 brings together Office 365, Windows 10 and Enterprise Mobility + Security.
"Microsoft 365 is where the world gets its best work done," said Joe Belfiore, corporate vice president, Microsoft. "With 135 million commercial monthly active users of Office 365 and nearly 700 million Windows 10 connected devices, Microsoft 365 helps developers reach people how and where they work."
Microsoft announced new opportunities that help developers create smarter ways for people to work, including new tools that give them the flexibility to design experiences in the languages and frameworks of their choice. Microsoft 365, combined with insights from the Microsoft Graph, empowers developers to build intelligent applications. New tools include the following:
- New and updated Microsoft Teams APIs in the Microsoft Graph and support for organization-specific applications in Teams allows developers to create tailored, intelligent experiences based on the needs of a business or industry. Companies can also publish custom apps to the Teams app store.
- Deeper SharePoint integration into Microsoft Teams enables people to pin a SharePoint page directly into channels to enable deeper collaboration. Developers can use modern script-based frameworks like React within their projects to add more pieces that can be organized within SharePoint pages.
- New support for Power BI Visualizations in Excel makes it possible for developers to extend the custom visuals created for Power BI to Excel.
- Updates to the Fluent Design System that help developers create engaging experiences with Microsoft's updated design language. With UWP XAML Islands, developers can access more capable, flexible, powerful XAML controls regardless of which UI stack they use - whether it's Windows Forms, WPF or native Win32.
- .NET Core 3.0, which allows developers to use the latest version of .NET and have their application run in a stand-alone .NET environment so they can build apps that don't impact their broader organizational infrastructure.
- MSIX, a complete containerization solution providing a simple way to convert large catalogs of applications. It inherits all the features from UWP, as well as a managed security model and support for both enterprise management and the Microsoft Store.
- New Azure Machine Learning and JavaScript custom functions that let developers and organizations create their own additions to the Excel catalog of formulas.
- Windows Machine Learning, a new platform that enables developers to develop machine learning models in the intelligent cloud and then deploy them offline and in high performance to the PC platform.
Microsoft also introduced a set of technology experiences that addresses a more mobile workforce:
- A new way to connect your phone to your PC with Windows 10 that enables instant access to text messages, photos and notifications.Imagine being able to quickly drag and drop your phone's photos into a document on your PC in one swift movement - without having to take your phone out of your pocket. This new experience will begin to roll out in the Windows Insider Program soon.
- The Microsoft Launcher app on Android will support enterprise customers including line-of-business app discovery and IT advisement on configuration. The Microsoft Launcher app will also support Timeline for cross-device app launching.
- On an iPhone or iPad, Microsoft Edge browsing sessions are included in the Timeline experience on a Windows 10 PC. Now Microsoft's customers will be able to access Timeline on an iPhone with Microsoft Edge.
- Microsoft is enabling any developer to reach and engage new audiences with Sets, an easier way to organize your information and get back to what you were doing. With Sets, what belongs together stays together, making it easier to create. For developers, UWP will work with Sets from the start, helping to keep customers engaged.
- Adaptive Cards now supported within Microsoft 365 enable developers to create interactive content in messages coming from their apps, bots or services into Teams and Outlook.
New Microsoft Store revenue share
Joe Belfioreannouinced that a new Microsoft Store fee structure coming later this year.
Starting later this year, consumer applications (not including games) sold in Microsoft Store will deliver to developers 95% of the revenue earned from the purchase of your application or any in-app products in your application, when a customer uses a deep link to get to and purchase your application. When Microsoft delivers you a customer through any other method, such as in a collection on Microsoft Store or any other owned Microsoft properties, and purchases your application, you will receive 85% of the revenue earned from the purchase of your application or any in-app products in your application
The new fee structure is applicable to purchases made on Windows 10 PCs, Windows Mixed Reality, Windows Phone and Surface Hub devices and excludes purchases on Xbox consoles.
The fee structure will be defined in detail in an upcoming revision to the App Developer Agreement later this year.
Alexa and Cortana Becoming Friends
Microsoft and Amazon.com may be competing with each other in the cloud-computing market, but in the realm of voice-controlled personal assistants, Cortana and Alexa are forging a friendship.
On Monday, the two companies showed the fruits of a collaboration announced last August to let Amazon's Alexa and Microsoft's Cortana talk to each other. In a demonstration, Cortana executive Megan Saunders used Alexa to order milk and then had the assistant wake Cortana to check her schedule and send email.
Then Alexa Senior Vice President Tom Taylor demonstrated using Cortana at work to check on his restaurant reservation before asking Cortana to raise Alexa to order an Uber. Alexa even declared that she likes Cortana, although neither assistant sang The Turtles' hit "Happy Together."
Both companies are trying to get their voice assistants installed in cars, and Amazon is striking deals with PC makers to pre-install the software, another encroachment on Cortana's patch in Windows.