Copilot Vision is an experimental AI-feature that, Microsoft promises, is going to revolutionize the online life of computer users.
Microsoft says that Copilot Vision sees the pages that users are on, reads the content of the pages, and is there for users when they have questions about current or past content, or want to have a discussion about it.
Browsing is no longer a “lonely experience with just you and all your tabs” according to Microsoft. How awfully nice.
Good news is that Copilot Vision is an opt-in experience. It is only available for select Pro subscribers at the time of writing and can be enabled under Copilot Labs. Furthermore, it will only work “on a select set of websites initially”.
Microsoft has come up with examples to demonstrate the usefulness of Copilot Vision.
- Use it to plan a day at the museum, by “pointing out all the information you need to know before you visit”.
- Tell you which products on a page match your needs and preferences.
- Help you learn new games, for instance Geoguessr.
And privacy?
Copilot Vision is opt-in, which means that users have to enable it before it starts monitoring activity.
Microsoft says that Copilot Vision data is only kept during sessions and deleted afterwards. This means that everything a user said during a session and the context is deleted. Copilots responses are logged, however, according to Microsoft to “improve safety systems”.
Closing Words
Is Copilot Vision a useful tool? Who is it for? The examples that Microsoft provided do not sound overly spectacular. Telling me which products I most likely like on a page? Maybe on a page with thousands or products and endless scrolling for a preselection, but otherwise?
There is also the question of trust.
- Do you want an AI to monitor your browsing, even if it is just for a session?
- Do you trust Microsoft to delete the data after the session?
- Do you believe that the answers that Microsoft logs do not contain personal information?
- That Copilot Vision won’t get enabled automatically, by error.
Now it is your turn. Could Copilot Vision be a helpful tool in the future? Or do you see issues and problems that it could cause first and foremost? Feel free to leave a comment down below.
I belive that the data they will obtain from this Copilot Vision will include all sorts of personal identifiably information. Think of the about the medical, financial, websites you use and log into, the security of the sessions for these sites. Do you trust them not to collect this data? Do you trust them to not have this running even when you turn it off?
First, stop calling it AI. A more accururate term would be VI. (virtual intelligence)
Second, until it and all the data it collects are stored and run locally and fully under my control, no forking way!
What is the definition of “personal information” does Microsoft use?
Things like:
1) The computer’s IP address?
2) Applications running running on the computer?
3) Extensions running in the web browser?
4) Your sex, race, gender, sexual orientation, political views, etc, etc?
All of the above, in combination or in part, could be used to “identify” a person…
The indication that the product will only work “on a select set of websites initially” suggests that the website domain will be logged (which illustrates issues with recording item 4 above!)
So… Copilot Vision is “opt-in” only? I don’t necessarily believe that. What about the infamous “nag screens” that are bound to infiltrate anything Microsoft thinks we plebians should be using. When do the nag screens start? [sarcasm intended]
Copilot is a virus/keylogger. You should treat it as one, regardless what Microsoft says.
Copilot is digital DEI and low-key racist by doing absolutely everything for the user. Zero human intelligence required. How wonderful for humanity, right? Right?
Just a thought, but perhaps they should have spent a little of those billions of USD on the question “what is the purpose of this”, instead of gifting it to NVIDIA and Mr Altman’s bank account and then coming up with purposes that read like a response to an LLM prompt. At this rate there will be a great number of upset Microsoft shareholders in a year or so after the announcement that the company will write off its billions in AI investment. And will no one think of the useless power plants?