Augmented Reality(AR) in Android
Co-Author: Alireza Eiji
Mobile applications have always been a place to do regular tasks like texting others and putting an item on your reminders list; but nowadays with the great power that has been put on smartphones, developers can push the boundaries of their application into reality.
What is Augmented Reality?
The main purpose of Augmented Reality (as you see in its name) is to add computer-modeled objects in the real world with the use of cameras and sensors. Your mobile phone gets information from the outside world, processes it, adds the objects you want to add to it, and shows the user the real-time image from the outside world but with some objects added to it.
Show me some examples, please
AR has made its way into so many industries and more and more developers are discovering its power and using it for many applications. Google Maps is one of them. Imagine you are searching for the nearby cafe you have found on Google Maps but since you are walking in a 3D world and seeing directions on a 2D screen, getting confused is somewhat regular. Google Maps has come up with an AR solution for this problem. It can direct you to your desired spot by adding direction flashes into the real world and you have to never be worried about getting lost. Look at the GIF below.
You telling me that I should build my AR app from the ground up?
Of course not. Tech companies have been releasing some good SDKs for developers to use. Since Google owns the Android OS, let’s look at its AR SDK called ARCore. ARCore introduces itself as follows:
ARCore is Google’s platform for building augmented reality experiences. Using different APIs, ARCore enables your phone to sense its environment, understand the world and interact with information. Some of the APIs are available across Android and iOS to enable shared AR experiences.
ARCore uses three key capabilities to integrate virtual content with the real world as seen through your phone’s camera:
Motion tracking allows the phone to understand and track its position relative to the world.
Environmental understanding allows the phone to detect the size and location of all type of surfaces: horizontal, vertical and angled surfaces like the ground, a coffee table or walls.
Light estimation allows the phone to estimate the environment’s current lighting conditions.
You can use this SDK for not only developing on the Android platform, but also on other platforms like iOS, Web, and Unity.
ARCore, like other AR tools, uses the camera input and combines it with the phone’s inertial sensors to determine both the position and orientation of the phone as it moves through space. It determines key points and remembers them, so when you move back your phone to that specific point, it can render the object on that place once again. It can also detect flat surfaces to put the objects on them.
Interesting. Now how can I build my AR app using ARCore?
In this text, we are going to explain ARCore using java on Android. But you can also see implementations of it on the ARCore website.
prerequisites
- Developing tool: Android Studio
- Basic understanding of Java and Android development
Enabling ARCore in your project
First of all, you should define whether you want your app to be defined as an “AR Required” app or an “AR Optional” app. Here is a list of differences between these two types of apps.
Add entries to your Manifest.xml file
for simplicity, we will assume that our app will be AR Optional. Therefore following codes will be for writing an AR Optional app.
Add the following entries to your Manifest.xml file.
<uses-permission android:name="android.permission.CAMERA" />
<application …>
<meta-data android:name="com.google.ar.core" android:value="optional" />
</application>Also, don't forget to set your minSdkVersion to 14 in build.gradle file since we are going the AR Optional way.
android {
defaultConfig {
…
minSdkVersion 14
}
}Of course, you need to add dependencies
Make sure your build.gradle contains Google’s Maven repository.
allprojects {
repositories {
google()
…
}
}and add this dependency to get the ARCore SDK.
dependencies {
…
implementation 'com.google.ar:core:1.23.0'
}How to know ARCore is supported during runtime?
Add the following snippet to your onCreate() method in ARActivity to know the availability.
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Enable AR-related functionality on ARCore supported devices only.
maybeEnableArButton();
…
}
void maybeEnableArButton() {
ArCoreApk.Availability availability = ArCoreApk.getInstance().checkAvailability(this);
if (availability.isTransient()) {
// Continue to query availability at 5Hz while compatibility is checked in the background.
new Handler().postDelayed(new Runnable() {
@Override
public void run() {
maybeEnableArButton();
}
}, 200);
}
if (availability.isSupported()) {
mArButton.setVisibility(View.VISIBLE);
mArButton.setEnabled(true);
} else { // The device is unsupported or unknown.
mArButton.setVisibility(View.INVISIBLE);
mArButton.setEnabled(false);
}
}You also need to get camera permision.
@Override
protected void onResume() {
super.onResume();
// ARCore requires camera permission to operate.
if (!CameraPermissionHelper.hasCameraPermission(this)) {
CameraPermissionHelper.requestCameraPermission(this);
return;
}
…
}Your app should also implement the following method.
@Override
public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] results) {
super.onRequestPermissionsResult(requestCode, permissions, results);
if (!CameraPermissionHelper.hasCameraPermission(this)) {
Toast.makeText(this, "Camera permission is needed to run this application", Toast.LENGTH_LONG)
.show();
if (!CameraPermissionHelper.shouldShowRequestPermissionRationale(this)) {
// Permission denied with checking "Do not ask again".
CameraPermissionHelper.launchPermissionSettings(this);
}
finish();
}
}Last but least, Check whether Google Play Services for AR is installed
Add the following snippet to your onResume() method.
// requestInstall(Activity, true) will triggers installation of
// Google Play Services for AR if necessary.
private boolean mUserRequestedInstall = true;
@Override
protected void onResume() {
super.onResume();
// Check camera permission.
…
// Ensure that Google Play Services for AR and ARCore device profile data are
// installed and up to date.
try {
if (mSession == null) {
switch (ArCoreApk.getInstance().requestInstall(this, mUserRequestedInstall)) {
case INSTALLED:
// Success: Safe to create the AR session.
mSession = new Session(this);
break;
case INSTALL_REQUESTED:
// When this method returns `INSTALL_REQUESTED`:
// 1. ARCore pauses this activity.
// 2. ARCore prompts the user to install or update Google Play
// Services for AR (market://details?id=com.google.ar.core).
// 3. ARCore downloads the latest device profile data.
// 4. ARCore resumes this activity. The next invocation of
// requestInstall() will either return `INSTALLED` or throw an
// exception if the installation or update did not succeed.
mUserRequestedInstall = false;
return;
}
}
} catch (UnavailableUserDeclinedInstallationException e) {
// Display an appropriate message to the user and return gracefully.
Toast.makeText(this, "TODO: handle exception " + e, Toast.LENGTH_LONG)
.show();
return;
} catch (…) {
…
return; // mSession remains null, since session creation has failed.
}
…
}Get the sample project and run it.
Clone the following URL from the official ARCore repository.
git clone https://github.com/google-ar/arcore-android-sdk.git
Look at these cute androids floating on the table. So what is the next step for you in the AR world. Go around and play with the sample code. Look at Google’s official documentation. Now that you have seen your first AR app, let’s see other SDKs and whether you should use them or ARCore is good.
