Facebook comes up with a new AI that is capable of predicting the future and helping in guiding activities. It is exciting to know, right? This is a new development in Facebook’s ongoing development of AR wearables. The new Anticipative Video Transformer or AVT, a new machine learning process that is outlined by Social Network which can predict future actions based on visual interpretation. The development can analyze activity and anticipate what action is likely to occur as a result.
Facebook says that AVT could be especially useful for applications such as an AR “action coach” or an AI assistant, by prompting someone that they may be about to make a mistake in completing a task or by reacting ahead of time with a helpful prompt for the next step in a task. For instance, AVT could warn someone that the pain they’re about to pick up is hot, based on the person’s previous interactions with the pan”.
Even though it sounds like a sci-fi movie that is capable of all-new smart home applications. When we talk about AR glasses, it can provide a range of useful pointers to aid people both at home and work in undertaking a wide variety of tasks.
Facebook trains the model to anticipate actions using three losses. First being, by classifying the features in the last frame of the video clip to predict labeled futuristic action. Secondly, they regress the intermediate frame feature to the features of the succeeding frames, which can help in understanding what comes next. And finally, it trains the model to classify intermediate actions. The model, along with three losses, predicts actions of future 10 percent to 30 percent more aptly than the other trained models with only bidirectional attention.
This model is going to be significant and will lead to new ways of guiding user actions by decreasing mistakes by predicting the future steps. Facebook takes the example of changing a car tire, with AR glasses helping in understanding the right direction, or it may also act as the reminder for morning routines, and anything based on visually assessing what and when you are doing things. When we also look at how Google Glass evolved to become a key tool in industrial workplaces, by providing in-view pointers and instructions for technical applications, the added potential for Facebook’s wearable AR devices is significant.
Let’s know how it can help
AVT can easily be rolled out autoregressive to predict longer into the future, which can help in long term planning tasks such as AR glasses observing that the person wearing them is changing a flat tire. It can predict steps for tasks and prompts the wearer to select the specific tool for the purposes. It can also help in self-supervised learning, the discovery of action schemas and boundaries, even for general action recognition in the task in a chronological manner.