The drag and drop feature for adding files to a website or attaching them to our emails just by dragging those from their location to the website application has become second nature to us. It is a feature that saves minutes of tedious browsing through the pop-up window, which does not let us switch to a different screen until the file is selected, or the user decides to quit. But imagine an extension of this feature wherein you could drag and drop real-world objects into your digital world, wouldn’t that ease your life? Augmented reality answers this, now enabling us to copy-paste real-world items into your digital screens just by few clicks.
Sounds even weird, right? Whenever the word Augmented Reality comes to our mind, we imagine Pokemon Go, animations showing us directions to our destination or Instagram filters that help people set a variety of stories. Until now, Augmented Reality has enhanced our real-world experience with various features within our screens within the digital world. To help check out how specific clothing suits us, how makeup cosmetics look, and much more. But this idea of bringing real-world objects into the digital world brings about a complete shift from our perspective and our interaction with Augmented Reality Apps.
Cyril Diagne — a designer, artist, and programmer took to Twitter and posted a video demonstrating his Augmented reality copy paste on photoshop. It contains a short description of his Augmented Reality App, the AR copy and paste prototype, which captured real-world objects and added those to his photoshop desktop screen.
The process of pasting real-world objects by clicking the snap of an object, getting rid of its background, and sharing it with your desktop is a tedious one. With this Augmented Reality App, it replaces this whole process by pointing your camera selecting the desired object, making it efficient by bypassing snapping, masking, saving, and switching windows for the user – making it very efficient AR copy paste photoshop tweak. The whole process is analogous to a pick-and-place robot, with the phone as the end effector or the claw grabbing the desired object after separating it from the background and placing it to the desired open workspace just by pointing the phone to the screen.
The Ingredients that make this happen!
AR copy and paste is an exciting concoction of Augmented Reality and ML technologies. The recipe contains two primary ingredients.
- Grabbing the object from its surroundings – Copying the object
- Positioning it in the right place – Pasting it in the right place
The secret sauce for copying the object, as Cyril Diagne mentions in his tweet, is the BASNet. This predict-refine architecture enables the identification of the real-world object and masks it out from its surrounding. Until date, various techniques have evolved for object detection focused on region accuracy but not on the boundary quality. Analogous to the human vision system, which has an effective attention mechanism for choosing essential information from visual scenes, this computer vision technique aims to model this mechanism through salient object detection. BASNet focuses and aims at accurately segmenting the pixels of prominent objects in an input image. BASNet achieves accurate salient object segmentation with high-quality boundaries.
Objects then are pasted into the working screen using screen-point, a Python library that converts camera centroid into screen coordinates using the OpenCV SIFT algorithm. D. Lowe, of the University of British Columbia, came up with, Scale Invariant Feature Transform (SIFT) algorithm. The algorithm can extract key points, compute its descriptors, and match corresponding key points between two images by identifying their nearest neighbors. This algorithm helps understand where the phone points are at the screen by matching the coordinates of the camera with the screen, the second ingredient allowing positioning of the image.
Latency, Scope of Improvements, Open Sourcing
The latency rate calculated by the developer for this operation was approximately around 2.5s for copying and 4s for pasting, reducing minutes of grind to a few seconds. Just a week after his first video of the prototype, the Greek programmer found even more impressive results about object detection using U^2-Net, a sequel of BASNet, when the authors of BASNet released its next version on 5th May. He also invites designers, programmers, and developers to contribute to AR copy paste as he believes that this process could be sped up and improved with a ton of ways, owing to which he has put up the code on GitHub for anyone to check out.
From Pasting Images to Pasting texts
AR copy and paste isn’t just an Augmented Reality copy paste photoshop tweak – it has a diversified range of applications and can be used in conjunction with a variety of apps like Adobe Creative Suite, Powerpoint, Keynote, MS Paint and Google Docs.
Imagine you have a presentation to give, and to make it impactful, you need to add real-world objects or attach images from textbooks. Instead of clicking a snap, masking it out in a different application, and then pasting it into slides with the help of AR copy and paste, all you’d need to do is scan the image and point it down to your slide and Voila!
The prototype cannot only be pasted on various workspace applications but can also copy printed texts from textbooks alongside images. By selecting printed prose, they can now be copied and affixed to your desired workspace, like Word and Photoshop, saving you strenuous efforts of typing the text all over again just by few clicks.
With a variety of useful applications like Augmented Reality copy-paste photoshop, AR Goggles, AR Online exhibitions, the field of Augmented Reality Technology seems promising.
- AR Copy Paste – Paste Printed Text!
With a diverse range of applications supporting AR copy paste and in the wake of use cases described by various groups of creatives, Diagne and his team were motivated to turn the prototype into a real-world application that anyone can use. As of now, you can visit https://arcopypaste.app for early access to Beta.
Seeing people so psyched to have early access to Beta, in the coming future, it would be a no-brainer to copy-paste real-world objects to your workspace by using such Augmented Reality-based applications.
What difference can it create for my business?
Augmented Reality can go all the way from changing the way your users interact with your system to enhancing certain features and making life simpler for them.
- In the fields of design, it can help reduce the inertia for users to translate hand-drawn or ancient illustration right into your workspace within seconds.
- For academia related tasks, AR copy and paste can help create a more effective presentation, automate bringing printed texts to your digital workspace
- Searching experience especially can be made more interactive, more fluid, broadening the user experience if we are able to copy real-world text, photos, audios, and paste them directly to search bars.
- It can create user-engagement: Games or applications can be made more user-engaging, just imagine using Augmented Reality for digital world Treasure-Hunt games wherein you have to copy-paste real-world objects.
- If your tech is made receptive to this technology like AR copypaste photoshop, it could be brought to different workspaces like printers or projectors, Augmented Reality can help print or project a real-world object within a few clicks and snaps.
Augmented Reality copy-paste, hence isn’t just an AR copy paste photoshop app but can have a myriad of applications and extensions. Just the way apps like Whatsapp, Swiggy, Amazon have become ingrained to our day to day lifestyle replacing traditional texting, Food orders, delivery. Augmented reality apps like Augmented Reality copy-paste won’t take much time to become second nature and again remind us that technologies have the answers to break our day-to-day faced limitations.
