page.title=3D Graphics
parent.title=RenderScript
parent.link=index.html
@jd:body
<div id="qv-wrapper">
<div id="qv">
<h2>In this document</h2>
<ol>
<li>
<a href="#creating-graphics-rs">Creating a Graphics Renderscript</a>
<ol>
<li><a href="#creating-native">Creating the native Renderscript file</a></li>
<li><a href="#creating-entry">Creating the Renderscript entry point class</a></li>
<li><a href="#creating-view">Creating the surface view</a></li>
<li><a href="#creating-activity">Creating the activity</a></li>
</ol>
</li>
<li>
<a href="#drawing">Drawing</a>
<ol>
<li><a href="#drawing-rsg">Drawing using the rsgDraw functions</a></li>
<li><a href="#drawing-mesh">Drawing with a mesh</a></li>
</ol>
</li>
<li>
<a href="#shaders">Shaders</a>
<ol>
<li><a href="#shader-bindings">Shader bindings</a></li>
<li><a href="#shader-sampler">Defining a sampler</a></li>
</ol>
</li>
</ol>
<h2>Related Samples</h2>
<ol>
<li><a href="{@docRoot}resources/samples/RenderScript/Balls/index.html">Balls</a></li>
<li><a href="{@docRoot}resources/samples/RenderScript/Fountain/index.html">Fountain</a></li>
<li><a href="{@docRoot}resources/samples/RenderScript/HelloWorld/index.html">Hello
World</a></li>
<li><a
href="{@docRoot}resources/samples/RenderScript/MiscSamples/index.html">Misc Samples</a></li>
</ol>
</div>
</div>
<p>RenderScript provides a number of graphics APIs for 3D rendering, both at the Android
framework level as well as at the native level. For instance, the Android framework APIs let you
create meshes and define shaders to customize the graphical rendering pipeline. The native
RenderScript graphics APIs lets you draw the actual meshes to render your scene. In general, you
will need to be familiar with APIs to appropriately render 3D graphics on an Android-powered
device.</p>
<h2 id="creating-graphics-rs">Creating a Graphics RenderScript</h2>
<p>Because of the various layers of code when writing a RenderScript application, it is useful to
create the following files for a scene that you want to render:</p>
<ul>
<li>The native RenderScript <code>.rs</code> file. This file contains the logic to do the
graphics rendering.</li>
<li>The RenderScript entry point class that allows your view to interact with the code defined
in the <code>.rs</code> file. This class contains a RenderScript object(instance of
<code>ScriptC_<em>renderscript_file</em></code>), which allows your Android framework code to
call the native RenderScript code. This class also creates the {@link
android.renderscript.RenderScriptGL} context object, which contains the current rendering state
of the RenderScript such as programs (vertex and fragment shaders, for example) that you want
to define and bind to the graphics pipeline. The context object attaches to the RenderScript
object (instance of <code><em>ScriptC_renderscript_file</em></code>) that does the rendering.
Our example names this class <code>HelloWorldRS</code>.</li>
<li>Create a class that extends {@link android.renderscript.RSSurfaceView} to provide a surface
to render on. If you want to implement callbacks from events inherited from {@link
android.view.View}, such as {@link android.view.View#onTouchEvent onTouchEvent()} and {@link
android.view.View#onKeyDown onKeyDown()}, do so in this class as well.</li>
<li>Create a class that is the main Activity class, like you would with any Android
application. This class sets your {@link android.renderscript.RSSurfaceView} as the content
view for this Activity.</li>
</ul>
<p>The following sections describe how to implement these three classes by using the HelloWorld
RenderScript sample that is provided in the SDK as a guide (some code has been modified from its
original form for simplicity).</p>
<h3 id="creating-native">Creating the native RenderScript file</h3>
<p>Your native RenderScript code resides in a <code>.rs</code> file in the
<code><project_root>/src/</code> directory. You can also define <code>.rsh</code> header
files. This code contains the logic to render your graphics and declares all necessary variables
and pointers. Every graphics <code>.rs</code> file generally contains the following items:</p>
<ul>
<li>A pragma (<code>#pragma rs java_package_name(<em>package.name</em>)</code>) that declares
the package name of the <code>.java</code> reflection of this RenderScript.</li>
<li>A pragma (<code>#pragma version(1)</code>) that declares the version of RenderScript that
you are using (1 is the only value for now).</li>
<li>A <code>#include</code> of the rs_graphics.rsh header file.</li>
<li>A <code>root()</code> function. This is the main worker function for your RenderScript and
calls RenderScript graphics APIs to draw meshes to the surface. This function is called every
time a frame refresh occurs, which is specified as its return value. A <code>0</code> specified
for the return value says to only render the frame when a property of the scene that you are
rendering changes. A non-zero positive integer specifies the refresh rate of the frame in
milliseconds.
<p class="note"><strong>Note:</strong> The RenderScript runtime makes its best effort to
refresh the frame at the specified rate. For example, if you are creating a live wallpaper
and set the return value to 50, the runtime renders the wallpaper at 20fps if it has just
enough or more resources to do so, and renders as fast as it can if it does not.</p>
<p>For more
information on using the RenderScript graphics functions, see the <a href=
"#drawing">Drawing</a> section.</p>
</li>
<li>An <code>init()</code> function. This allows you to do any initialization of your
RenderScript before the <code>root()</code> function runs, such as initializing variables. This
function runs once and is called automatically when the RenderScript starts, before anything
else in your RenderScript. Creating this function is optional.</li>
<li>Any variables, pointers, and structures that you wish to use in your RenderScript code (can
be declared in <code>.rsh</code> files if desired)</li>
</ul>
<p>The following code shows how the <code>helloworld.rs</code> file is implemented:</p>
<pre>
#pragma version(1)
// Tell which java package name the reflected files should belong to
#pragma rs java_package_name(com.android.rs.helloworld)
// Built-in header with graphics APIs
#include "rs_graphics.rsh"
// gTouchX and gTouchY are variables that are reflected for use
// by the Android framework API. This RenderScript uses them to be notified of touch events.
int gTouchX;
int gTouchY;
// This is invoked automatically when the script is created and initializes the variables
// in the Android framework layer as well.
void init() {
gTouchX = 50.0f;
gTouchY = 50.0f;
}
int root(int launchID) {
// Clear the background color
rsgClearColor(0.0f, 0.0f, 0.0f, 0.0f);
// Tell the runtime what the font color should be
rsgFontColor(1.0f, 1.0f, 1.0f, 1.0f);
// Introuduce ourselves to the world by drawing a greeting
// at the position user touched on the screen
rsgDrawText("Hello World!", gTouchX, gTouchY);
// Return value tells RS roughly how often to redraw
// in this case 20 ms
return 20;
}
</pre>
<h3 id="creating-entry">Creating the RenderScript entry point class</h3>
<p>When you create a RenderScript (<code>.rs</code>) file, it is helpful to create a
corresponding Android framework class that is an entry point into the <code>.rs</code> file. In
this entry point class, you create a RenderScript object by instantiating a
<code>ScriptC_<em>rs_filename</em></code> and binding it to the RenderScript context. The
RenderScript object is attached to the RenderScript bytecode, which is platform-independent and
gets compiled on the device when the RenderScript application runs. Both the
<code>ScriptC_<em>rs_filename</em></code> class and bytecode is generated by the Android build
tools and is packaged with the <code>.apk</code> file. The bytecode file is located in the
<code><project_root>/res/raw/</code> directory and is named <code>rs_filename.bc</code>.
You refer to the bytecode as a resource (<code>R.raw.<em>rs_filename</em></code>). when creating
the RenderScript object..</p>
<p>You then bind the RenderScript object to the RenderScript context, so that the surface view
knows what code to use to render graphics. The following code shows how the
<code>HelloWorldRS</code> class is implemented:</p>
<pre>
package com.android.rs.helloworld;
import android.content.res.Resources;
import android.renderscript.*;
public class HelloWorldRS {
//context and resources are obtained from RSSurfaceView, which calls init()
private Resources mRes;
private RenderScriptGL mRS;
//Declare the RenderScript object
private ScriptC_helloworld mScript;
public HelloWorldRS() {
}
/**
* This provides us with the RenderScript context and resources
* that allow us to create the RenderScript object
*/
public void init(RenderScriptGL rs, Resources res) {
mRS = rs;
mRes = res;
initRS();
}
/**
* Calls native RenderScript functions (set_gTouchX and set_gTouchY)
* through the reflected layer class ScriptC_helloworld to pass in
* touch point data.
*/
public void onActionDown(int x, int y) {
mScript.set_gTouchX(x);
mScript.set_gTouchY(y);
}
/**
* Binds the RenderScript object to the RenderScript context
*/
private void initRS() {
//create the RenderScript object
mScript = new ScriptC_helloworld(mRS, mRes, R.raw.helloworld);
//bind the RenderScript object to the RenderScript context
mRS.bindRootScript(mScript);
}
}
</pre>
<h3 id="creating-view">Creating the surface view</h3>
<p>To create a surface view to render graphics on, create a class that extends {@link
android.renderscript.RSSurfaceView}. This class also creates a RenderScript context object
({@link android.renderscript.RenderScriptGL} and passes it to the Rendscript entry point class to
bind the two. The following code shows how the <code>HelloWorldView</code> class is
implemented:</p>
<pre>
package com.android.rs.helloworld;
import android.renderscript.RSSurfaceView;
import android.renderscript.RenderScriptGL;
import android.content.Context;
import android.view.MotionEvent;
public class HelloWorldView extends RSSurfaceView {
// RenderScript context
private RenderScriptGL mRS;
// RenderScript entry point object that does the rendering
private HelloWorldRS mRender;
public HelloWorldView(Context context) {
super(context);
initRS();
}
private void initRS() {
if (mRS == null) {
// Initialize RenderScript with default surface characteristics.
RenderScriptGL.SurfaceConfig sc = new RenderScriptGL.SurfaceConfig();
//Create the RenderScript context
mRS = createRenderScriptGL(sc);
// Create an instance of the RenderScript entry point class
mRender = new HelloWorldRS();
// Call the entry point class to bind it to this context
mRender.init(mRS, getResources());
}
}
/**
* Rebind everything when the window becomes attached
*/
protected void onAttachedToWindow() {
super.onAttachedToWindow();
initRS();
}
/**
* Stop rendering when window becomes detached
*/
protected void onDetachedFromWindow() {
// Handle the system event and clean up
mRender = null;
if (mRS != null) {
mRS = null;
destroyRenderScriptGL();
}
}
/**
* Use callbacks to relay data to RenderScript entry point class
*/
public boolean onTouchEvent(MotionEvent ev) {
// Pass touch events from the system to the rendering script
if (ev.getAction() == MotionEvent.ACTION_DOWN) {
mRender.onActionDown((int)ev.getX(), (int)ev.getY());
return true;
}
return false;
}
}
</pre>
<h3 id="creating-activity">Creating the Activity</h3>
<p>Applications that use RenderScript still adhere to activity lifecyle, and are part of the same
view hierarchy as traditional Android applications, which is handled by the Android VM. This
Activity class sets its view to be the {@link android.renderscript.RSSurfaceView} and handles
lifecycle callback events appropriately. The following code shows how the <code>HelloWorld</code>
class is implemented:</p>
<pre>
public class HelloWorldActivity extends Activity {
//Custom view to use with RenderScript
private HelloWorldView view;
public void onCreate(Bundle icicle) {
super.onCreate(icicle);
// Create surface view and set it as the content of our Activity
mView = new HelloWorldView(this);
setContentView(view);
}
protected void onResume() {
// Ideally an app should implement onResume() and onPause()
// to take appropriate action when the activity loses focus
super.onResume();
view.resume();
}
protected void onPause() {
// Ideally an app should implement onResume() and onPause()
// to take appropriate action when the activity loses focus
super.onPause();
view.pause();
}
}
</pre>
<h2 id="drawing">Drawing</h2>
<p>The following sections describe how to use the graphics functions to draw with Renderscript.</p>
<h3 id="drawing-rsg">Drawing using the rsgDraw functions</h3>
<p>The native RenderScript APIs provide a few convenient functions to easily draw a polygon to
the screen. You call these in your <code>root()</code> function to have them render to the
surface view. These functions are available for simple drawing and should not be used for complex
graphics rendering:</p>
<ul>
<li><code>rsgDrawRect()</code>: Sets up a mesh and draws a rectangle to the screen. It uses the
top left vertex and bottom right vertex of the rectangle to draw.</li>
<li><code>rsgDrawQuad()</code>: Sets up a mesh and draws a quadrilateral to the screen.</li>
<li><code>rsgDrawQuadTexCoords()</code>: Sets up a mesh and draws a textured quadrilateral to
the screen.</li>
</ul>
<h3 id="drawing-mesh">Drawing with a mesh</h3>
<p>When you want to draw complex shapes and textures to the screen, instantiate a {@link
android.renderscript.Mesh} and draw it to the screen with <code>rsgDrawMesh()</code>. A {@link
android.renderscript.Mesh} is a collection of allocations that represent vertex data (positions,
normals, texture coordinates) and index data such as triangles and lines. You can build a Mesh in
three different ways:</p>
<ul>
<li>Build the mesh with the {@link android.renderscript.Mesh.TriangleMeshBuilder} class, which
allows you to specify a set of vertices and indices for each triangle that you want to draw.
The downside of doing it this way is there is no way to specify the vertices in your native
RenderScript code.</li>
<li>Build the mesh using an {@link android.renderscript.Allocation} or a set of {@link
android.renderscript.Allocation}s with the {@link android.renderscript.Mesh.AllocationBuilder}
class. This allows you to build a mesh with vertices already stored in memory, which allows you
to set the vertices in native or Android code.</li>
<li>Build the mesh with the {@link android.renderscript.Mesh.Builder} class. This is a
convenience method for when you know what data types you want to use to build your mesh, but
don't want to make separate memory allocations like with {@link
android.renderscript.Mesh.AllocationBuilder}. You can specify the types that you want and this
mesh builder automatically creates the memory allocations for you.</li>
</ul>
<p>To create a mesh using the {@link android.renderscript.Mesh.TriangleMeshBuilder}, you need to
supply it with a set of vertices and the indices for the vertices that comprise the triangle. For
example, the following code specifies three vertices, which are added to an internal array,
indexed in the order they were added. The call to {@link
android.renderscript.Mesh.TriangleMeshBuilder#addTriangle addTriangle()} draws the triangle with
vertex 0, 1, and 2 (the vertices are drawn counter-clockwise).</p>
<pre>
int float2VtxSize = 2;
Mesh.TriangleMeshBuilder triangle = new Mesh.TriangleMeshBuilder(renderscriptGL,
float2VtxSize, Mesh.TriangleMeshBuilder.COLOR);
triangles.addVertex(300.f, 300.f);
triangles.addVertex(150.f, 450.f);
triangles.addVertex(450.f, 450.f);
triangles.addTriangle(0 , 1, 2);
Mesh smP = triangle.create(true);
script.set_mesh(smP);
</pre>
<p>To draw a mesh using the {@link android.renderscript.Mesh.AllocationBuilder}, you need to
supply it with one or more allocations that contain the vertex data:</p>
<pre>
Allocation vertices;
...
Mesh.AllocationBuilder triangle = new Mesh.AllocationBuilder(mRS);
smb.addVertexAllocation(vertices.getAllocation());
smb.addIndexSetType(Mesh.Primitive.TRIANGLE);
Mesh smP = smb.create();
script.set_mesh(smP);
</pre>
<p>In your native RenderScript code, draw the built mesh to the screen:</p>
<pre>
rs_mesh mesh;
...
int root(){
...
rsgDrawMesh(mesh);
...
return 0; //specify a non zero, positive integer to specify the frame refresh.
//0 refreshes the frame only when the mesh changes.
}
</pre>
<h2 id="shaders">Shaders</h2>
<p>You can attach four program objects to the {@link android.renderscript.RenderScriptGL} context
to customize the rendering pipeline. For example, you can create vertex and fragment shaders in
GLSL or build a raster program object with provided methods without writing GLSL code. The four
program objects mirror a traditional graphical rendering pipeline:</p>
<table>
<tr>
<th>Android Object Type</th>
<th>RenderScript Native Type</th>
<th>Description</th>
</tr>
<tr>
<td>{@link android.renderscript.ProgramVertex}</td>
<td>rs_program_vertex</td>
<td>
<p>The RenderScript vertex program, also known as a vertex shader, describes the stage in
the graphics pipeline responsible for manipulating geometric data in a user-defined way.
The object is constructed by providing RenderScript with the following data:</p>
<ul>
<li>An Element describing its varying inputs or attributes</li>
<li>GLSL shader string that defines the body of the program</li>
<li>a Type that describes the layout of an Allocation containing constant or uniform
inputs</li>
</ul>
<p>Once the program is created, bind it to the {@link android.renderscript.RenderScriptGL}
graphics context by calling {@link android.renderscript.RenderScriptGL#bindProgramVertex
bindProgramVertex()}. It is then used for all subsequent draw calls until you bind a new
program. If the program has constant inputs, the user needs to bind an allocation
containing those inputs. The allocation's type must match the one provided during creation.
The RenderScript library then does all the necessary plumbing to send those constants to
the graphics hardware. Varying inputs to the shader, such as position, normal, and texture
coordinates are matched by name between the input Element and the Mesh object being drawn.
The signatures don't have to be exact or in any strict order. As long as the input name in
the shader matches a channel name and size available on the mesh, the run-time would take
care of connecting the two. Unlike OpenGL, there is no need to link the vertex and fragment
programs.</p>
<p>To bind shader constructs to the Program, declare a struct containing the necessary
shader constants in your native RenderScript code. This struct is generated into a
reflected class that you can use as a constant input element during the Program's creation.
It is an easy way to create an instance of this struct as an allocation. You would then
bind this Allocation to the Program and the RenderScript system sends the data that is
contained in the struct to the hardware when necessary. To update shader constants, you
change the values in the Allocation and notify the native RenderScript code of the
change.</p>
</td>
</tr>
<tr>
<td>{@link android.renderscript.ProgramFragment}</td>
<td>rs_program_fragment</td>
<td>
<p>The RenderScript fragment program, also known as the fragment shader, is responsible for
manipulating pixel data in a user-defined way. It's constructed from a GLSL shader string
containing the program body, textures inputs, and a Type object describing the constants
used by the program. Like the vertex programs, when an allocation with constant input
values is bound to the shader, its values are sent to the graphics program automatically.
Note that the values inside the allocation are not explicitly tracked. If they change
between two draw calls using the same program object, notify the runtime of that change by
calling rsgAllocationSyncAll so it could send the new values to hardware. Communication
between the vertex and fragment programs is handled internally in the GLSL code. For
example, if the fragment program is expecting a varying input called varTex0, the GLSL code
inside the program vertex must provide it.</p>
<p>To bind shader constants to this program, declare a struct containing the necessary
shader constants in your native RenderScript code. This struct is generated into a
reflected class that you can use as a constant input element during the Program's creation.
It is an easy way to create an instance of this struct as an allocation. You would then
bind this Allocation to the Program and the RenderScript system sends the data that is
contained in the struct to the hardware when necessary. To update shader constants, you
change the values in the Allocation and notify the native RenderScript code of the
change.</p>
</td>
</tr>
<tr>
<td>{@link android.renderscript.ProgramStore}</td>
<td>rs_program_store</td>
<td>The RenderScript ProgramStore contains a set of parameters that control how the graphics
hardware writes to the framebuffer. It could be used to enable and disable depth writes and
testing, setup various blending modes for effects like transparency and define write masks
for color components.</td>
</tr>
<tr>
<td>{@link android.renderscript.ProgramRaster}</td>
<td>rs_program_raster</td>
<td>Program raster is primarily used to specify whether point sprites are enabled and to
control the culling mode. By default back faces are culled.</td>
</tr>
</table>
<p>The following example defines a vertex shader in GLSL and binds it to the RenderScript:</p>
<pre>
private RenderScriptGL glRenderer; //rendering context
private ScriptField_Point mPoints; //vertices
private ScriptField_VpConsts mVpConsts; //shader constants
...
ProgramVertex.Builder sb = new ProgramVertex.Builder(glRenderer);
String t = "varying vec4 varColor;\n" +
"void main() {\n" +
" vec4 pos = vec4(0.0, 0.0, 0.0, 1.0);\n" +
" pos.xy = ATTRIB_position;\n" +
" gl_Position = UNI_MVP * pos;\n" +
" varColor = vec4(1.0, 1.0, 1.0, 1.0);\n" +
" gl_PointSize = ATTRIB_size;\n" +
"}\n";
sb.setShader(t);
sb.addConstant(mVpConsts.getType());
sb.addInput(mPoints.getElement());
ProgramVertex pvs = sb.create();
pvs.bindConstants(mVpConsts.getAllocation(), 0);
glRenderer.bindProgramVertex(pvs);
</pre>
<p>The <a href=
"{@docRoot}resources/samples/RenderScript/MiscSamples/src/com/example/android/rs/miscsamples/RsRenderStatesRS.html">
RsRenderStatesRS</a> sample has many examples on how to create a shader without writing GLSL.</p>
<h3 id="shader-bindings">Shader bindings</h3>
<p>You can also set four pragmas that control the shaders' default bindings to the {@link
android.renderscript.RenderScriptGL} context when the script is executing:</p>
<ul>
<li>stateVertex</li>
<li>stateFragment</li>
<li>stateRaster</li>
<li>stateStore</li>
</ul>
<p>The possible values for each pragma are <code>parent</code> or <code>default</code>. Using
<code>default</code> binds the shaders to the graphical context with the system defaults. The
default shader is defined below:</p>
<pre>
("varying vec4 varColor;\n");
("varying vec2 varTex0;\n");
("void main() {\n");
(" gl_Position = UNI_MVP * ATTRIB_position;\n");
(" gl_PointSize = 1.0;\n");
(" varColor = ATTRIB_color;\n");
(" varTex0 = ATTRIB_texture0;\n");
("}\n");
</pre>
<p>Using <code>parent</code> binds the shaders in the same manner as it is bound in the calling
script. If this is the root script, the parent state is taken from the bind points that are set
by the {@link android.renderscript.RenderScriptGL} bind methods.</p>
<p>For example, you can define this at the top of your native graphics RenderScript code to have
the Vertex and Store shaders inherent the bind properties from their parent scripts:</p>
<pre>
#pragma stateVertex(parent)
#pragma stateStore(parent)
</pre>
<h3 id="shader-sampler">Defining a sampler</h3>
<p>A {@link android.renderscript.Sampler} object defines how data is extracted from textures.
Samplers are bound to Program objects (currently only a Fragment Program) alongside the texture
whose sampling they control. These objects are used to specify such things as edge clamping
behavior, whether mip-maps are used, and the amount of anisotropy required. There might be
situations where hardware does not support the desired behavior of the sampler. In these cases,
the runtime attempts to provide the closest possible approximation. For example, the user
requested 16x anisotropy, but only 8x was set because it's the best available on the
hardware.</p>
<p>The <a href=
"{@docRoot}resources/samples/RenderScript/MiscSamples/src/com/example/android/rs/miscsamples/RsRenderStatesRS.html">
RsRenderStatesRS</a> sample has many examples on how to create a sampler and bind it to a
Fragment program.</p>
</body>
</html>