Author Topic: Texture2D.readpixels on Android with scaling  (Read 15374 times)

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Texture2D.readpixels on Android with scaling
« on: October 03, 2014, 05:00:10 AM »
Hi,

I'm trying to make a screenshot of an object in my GUI.
I'm using this code:
  1. public class TakeScreenShot : MonoBehaviour
  2. {
  3.         public int      startX,
  4.                                 startY,
  5.                                 width,
  6.                                 height;
  7.  
  8.         private IEnumerator ScreenShot()
  9.         {
  10.                 Debug.Log ("Grabbing texture!");
  11.                 Texture2D tex = new Texture2D(width, height, TextureFormat.RGB24, false);
  12.                 yield return new WaitForEndOfFrame();
  13.                 tex.ReadPixels(new Rect(startX, startY, width, height), 0, 0);
  14.                 tex.Apply();
  15.                
  16.                 Destroy(tex);
  17.                 byte[] bytes = tex.EncodeToPNG();
  18.                
  19.                 File.WriteAllBytes(Application.persistentDataPath + "/Banknote.png", bytes);
  20.                 StartCoroutine(ScreenshotManager.SaveExisting( bytes, "banknote"));
  21.         }
  22.  
  23.         public void CaptureImage(bool send)
  24.         {
  25.                 StartCoroutine("ScreenShot");
  26.         }
  27.  
  28.         public void SetCameraSettings(int x, int y, int width, int height)
  29.         {
  30.                 startX = x;
  31.                 startY = y;
  32.                 this.width = width;
  33.                 this.height = height;
  34.         }
  35. }
  36.  

The screenshot i'm taking is of the banknote (beige one in the picture below):



Now since i'm making this App on Android NGUI Scales everything on the resolution (and it works perfectly). But here's where the problem rises. Since the GUI scales on resolution the width of the screen changes (banknote stays the same width and height). What i'd like to do is calculate the difference between the left edge of the screen and the left edge of the banknote and the top edge of the screen and the top edge of the banknote. See the picture below:



The value that comes out of that calculation should function as the startX and startY position of the readpixels function. What is the best way to calculate this?

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #1 on: October 04, 2014, 07:43:55 AM »
UIRect.worldCorners tells you the 4 corners of any widget in world space. Use camera.WorldToScreenSpace to convert them to something usable for yourself. The corner you care about is corner[1] (which is top-left).

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #2 on: October 06, 2014, 02:42:35 AM »
UIRect.worldCorners tells you the 4 corners of any widget in world space. Use camera.WorldToScreenSpace to convert them to something usable for yourself. The corner you care about is corner[1] (which is top-left).

Hi ArenMook,

Thanks for your reply!

So far i've done the following:
  1. public void SetCameraSettings()
  2. {
  3.                 //The top left corner of the banknote i'm trying to grab
  4.                 Vector3 bankNoteWorldCorner = camera.WorldToScreenPoint(banknote.worldCorners[1]);
  5.  
  6.                 //Backgroud is white sliced sprite covering the whole screen
  7.                 Vector3 backgroundWorldCorner = camera.WorldToScreenPoint(background.worldCorners[1]);
  8.  
  9.                 Debug.Log ("BankNoteWC: " + bankNoteWorldCorner.ToString() + " backgroundWC: " + backgroundWorldCorner.ToString());
  10.  
  11.                 Debug.Log ("Starting to read pixels at: " + (backgroundWorldCorner.x + bankNoteWorldCorner.x));
  12.  
  13.                 startX = (int) (bankNoteWorldCorner.x);
  14.                 startY = (int) (bankNoteWorldCorner.y);
  15.  
  16.  
  17.                 //this.width = width;
  18.                 //this.height = height;
  19. }
  20.  

In my thoughts I don't even need the background world corner since I know the starting world corner of the banknote. But this only captures a grey area in a PNG.

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #3 on: October 06, 2014, 10:25:04 AM »
I see no code there that does any capturing. I also don't see what 'camera' is. Which camera is it? It needs to be the camera that draws your UI to begin with. Use ReadPixels afterwards.

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #4 on: November 04, 2014, 09:58:05 AM »
I see no code there that does any capturing. I also don't see what 'camera' is. Which camera is it? It needs to be the camera that draws your UI to begin with. Use ReadPixels afterwards.

Hi ArenMook,

Sorry for the really really late reply been a bit busy lately. But here's the full code of my TakeScreenShot class. The script is attached to the NGUI camera which draws the UI.
  1. using UnityEngine;
  2. using System.Collections;
  3. using System.IO;
  4.  
  5. public class TakeScreenShot : MonoBehaviour
  6. {
  7.  
  8.         public UISprite background;
  9.         public UITexture banknote;
  10.  
  11.         public int      startX,
  12.                                 startY,
  13.                                 width,
  14.                                 height;
  15.  
  16.         private void Start()
  17.         {
  18.                 SetCameraSettings();
  19.         }
  20.  
  21.  
  22.         private IEnumerator ScreenShot()
  23.         {
  24.                 Debug.Log ("Grabbing texture!");
  25.                 Texture2D tex = new Texture2D(width, height, TextureFormat.RGB24, false);
  26.                 yield return new WaitForEndOfFrame();
  27.                 tex.ReadPixels(new Rect(startX, startY, width, height), 0, 0);
  28.                 tex.Apply();
  29.                
  30.                 Destroy(tex);
  31.                 byte[] bytes = tex.EncodeToPNG();
  32.                
  33.                 File.WriteAllBytes(Application.persistentDataPath + "/Banknote.png", bytes);
  34.                 StartCoroutine(ScreenshotManager.SaveExisting( bytes, "banknote"));
  35.         }
  36.  
  37.         public void CaptureImage(bool send)
  38.         {
  39.                 StartCoroutine("ScreenShot");
  40.         }
  41.  
  42.         public void SetCameraSettings()
  43.         {
  44.                 //The top left corner of the banknote i'm trying to grab
  45.                 Vector3 bankNoteWorldCorner = camera.WorldToScreenPoint(banknote.worldCorners[1]);
  46.  
  47.                 //Backgroud is white sliced sprite covering the whole screen
  48.                 Vector3 backgroundWorldCorner = camera.WorldToScreenPoint(background.worldCorners[1]);
  49.  
  50.                 Debug.Log ("BankNoteWC: " + bankNoteWorldCorner.ToString() + " backgroundWC: " + backgroundWorldCorner.ToString());
  51.  
  52.                 Debug.Log ("Starting to read pixels at: " + (backgroundWorldCorner.x + bankNoteWorldCorner.x));
  53.  
  54.                 startX = (int) (bankNoteWorldCorner.x);
  55.                 startY = (int) (bankNoteWorldCorner.y);
  56.  
  57.  
  58.                 //this.width = width;
  59.                 //this.height = height;
  60.         }
  61. }
  62.  

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #5 on: November 04, 2014, 10:51:32 AM »
Not sure about that. Does ReadPixels even work properly on Android? To take screenshots you typically should be using Application.CaptureScreenshot.

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #6 on: November 06, 2014, 04:00:01 AM »
Not sure about that. Does ReadPixels even work properly on Android? To take screenshots you typically should be using Application.CaptureScreenshot.

It does work on Android, but the scaling still poses a problem. But if i start using Application.CaptureScreenshot, how would I then cut out the part I need instead of saving the whole screen?

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #7 on: November 07, 2014, 01:32:44 PM »
The screenshot would be the size of your screen, so in pixels. Same math applies.

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #8 on: November 10, 2014, 04:57:28 AM »
The screenshot would be the size of your screen, so in pixels. Same math applies.

True, but then the problem still exists, how will you know where the object is where I want to take a picture off when the screen scales to a different resolution?

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #9 on: November 10, 2014, 08:57:49 PM »
Math -- you were doing it -- worldCorners, WorldToScreenPoint. Although whether you were the right corner or not, I'm not sure. [1] is top-left, but only windows coordinates are top-left based. OSX and graphics are both bottom-left based, for example.

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #10 on: November 20, 2014, 05:53:15 AM »
Math -- you were doing it -- worldCorners, WorldToScreenPoint. Although whether you were the right corner or not, I'm not sure. [1] is top-left, but only windows coordinates are top-left based. OSX and graphics are both bottom-left based, for example.

True, but that brings us back to my initial post. I couldn't really get it working, that's why I was asking support.

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #11 on: November 20, 2014, 08:35:59 PM »
I don't know what coordinates are needed for readpixels, as well as when it should be used. I've never used this function myself, and it's a Unity question which is why I can't be of much use. From NGUI's side, the coordinates are trivial, and that's what I mentioned -- worldCorners, then WorldToScreenPoint.

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #12 on: November 26, 2014, 10:42:21 AM »
I don't know what coordinates are needed for readpixels, as well as when it should be used. I've never used this function myself, and it's a Unity question which is why I can't be of much use. From NGUI's side, the coordinates are trivial, and that's what I mentioned -- worldCorners, then WorldToScreenPoint.

I've tried to find the TopLeft corner of the banknote in the space used by Texture2D.readPixels (uses pixel space). But I can't get the right coordinates to begin the readpixels function with the banknote.worldcorners function. I've tried setting the cube's localposition and position.

My code:

  1. using UnityEngine;
  2. using System.Collections;
  3. using System.IO;
  4.  
  5. public class TakeScreenShot : MonoBehaviour
  6. {
  7.         //The banknote UITexture
  8.         public UITexture banknote;
  9.  
  10.         //Camera uses by NGUI
  11.         public UICamera cam;
  12.  
  13.         private IEnumerator ScreenShot()
  14.         {
  15.                 Debug.Log ("Grabbing texture!");
  16.  
  17.                 Vector3 bVector = new Vector3(banknote.worldCorners[1].x, banknote.worldCorners[1].y, 0);
  18.                 Vector3 scaleVector = new Vector3(0.1f, 0.1f, 0.1f);
  19.  
  20.                 GameObject obj01 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  21.                 obj01.transform.position = cam.camera.ScreenToWorldPoint(bVector);
  22.                 obj01.transform.localScale = scaleVector;
  23.                 obj01.renderer.material.color = Color.blue;
  24.  
  25.                 GameObject obj02 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  26.                 obj02.transform.position = cam.camera.ScreenToViewportPoint(bVector);
  27.                 obj02.transform.localScale = scaleVector;
  28.                 obj02.renderer.material.color = Color.red;
  29.  
  30.                 GameObject obj03 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  31.                 obj03.transform.position = cam.camera.WorldToScreenPoint(bVector);
  32.                 obj03.transform.localScale = scaleVector;
  33.                 obj03.renderer.material.color = Color.green;
  34.  
  35.                 GameObject obj04 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  36.                 obj04.transform.position = cam.camera.WorldToViewportPoint(bVector);
  37.                 obj04.transform.localScale = scaleVector;
  38.                 obj04.renderer.material.color = Color.yellow;
  39.  
  40.                 GameObject obj05 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  41.                 obj05.transform.localPosition = cam.camera.ScreenToWorldPoint(bVector);
  42.                 obj05.transform.localScale = scaleVector;
  43.                 obj05.renderer.material.color = Color.cyan;
  44.  
  45.                 GameObject obj06 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  46.                 obj06.transform.localPosition = cam.camera.ScreenToViewportPoint(bVector);
  47.                 obj06.transform.localScale = scaleVector;
  48.                 obj06.renderer.material.color = Color.magenta;
  49.  
  50.                 GameObject obj07 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  51.                 obj07.transform.localPosition = cam.camera.WorldToScreenPoint(bVector);
  52.                 obj07.transform.localScale = scaleVector;
  53.                 obj07.renderer.material.color = Color.white;
  54.  
  55.                 GameObject obj08 = GameObject.CreatePrimitive(PrimitiveType.Cube);
  56.                 obj08.transform.localPosition = cam.camera.WorldToViewportPoint(bVector);
  57.                 obj08.transform.localScale = scaleVector;
  58.                 obj08.renderer.material.color = Color.grey;
  59.  
  60.                 Texture2D tex = new Texture2D(banknote.width, banknote.height, TextureFormat.RGB24, false);
  61.                 yield return new WaitForEndOfFrame();
  62.  
  63.                 tex.ReadPixels(new Rect(banknote.transform.position.x, banknote.transform.position.y, banknote.width, banknote.height), 0, 0);
  64.                 tex.Apply();
  65.  
  66.                 Destroy(tex);
  67.                 byte[] bytes = tex.EncodeToPNG();
  68.                
  69.                 File.WriteAllBytes(Application.persistentDataPath + "/Banknote.png", bytes);
  70.                 StartCoroutine(ScreenshotManager.SaveExisting( bytes, "banknote"));
  71.         }
  72.  
  73.         public void CaptureImage(bool send)
  74.         {
  75.                 StartCoroutine("ScreenShot");
  76.         }
  77. }
  78.  

And the result:


As you can see none of the cubes are on the topleft position of the banknote...

The question is how do I calculate the TopLeft corner of the banknote object in pixel space.

Check the Unity3D Api on ReadPixels:
http://docs.unity3d.com/ScriptReference/Texture2D.ReadPixels.html
« Last Edit: November 26, 2014, 10:52:31 AM by Tripwire »

ArenMook

  • Administrator
  • Hero Member
  • *****
  • Thank You
  • -Given: 337
  • -Receive: 1171
  • Posts: 22,128
  • Toronto, Canada
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #13 on: November 26, 2014, 08:39:54 PM »
Look closer at what you're doing:

Vector3 bVector = new Vector3(banknote.worldCorners[1].x, banknote.worldCorners[1].y, 0);

obj01.transform.position = cam.camera.ScreenToWorldPoint(bVector);

It's already in world coordinates, yet you are converting it from screen to world.

All you needed to do was:

obj01.transform.position = banknote.worldCorners[1];

Tripwire

  • Full Member
  • ***
  • Thank You
  • -Given: 9
  • -Receive: 0
  • Posts: 163
    • View Profile
Re: Texture2D.readpixels on Android with scaling
« Reply #14 on: November 30, 2014, 06:31:26 PM »
Look closer at what you're doing:

Vector3 bVector = new Vector3(banknote.worldCorners[1].x, banknote.worldCorners[1].y, 0);

obj01.transform.position = cam.camera.ScreenToWorldPoint(bVector);

It's already in world coordinates, yet you are converting it from screen to world.

All you needed to do was:

obj01.transform.position = banknote.worldCorners[1];

Right! Silly me :) Now I only need to get the width of the object in WorldSpace (which is Pixel space I guess). How can I get the Width and Height in the correct space? I tried using the top left and top right values of the WorldCorners but when calculating they always return negative (using + and -) and ReadPixels can't work with negative values.