Images and Scaling with Windows Runtime #WinRT #WPDev #Windev

With the advent of Windows Phone 8.1, developers have the option of using WinRT native XAML stack as opposed to the Silverlight XAML the only option with early Windows Phone iterations.

One of the differences between WinRT XAML and Silverlight XAML is scaling and like with fonts, & images, scaling can have a significant effect on the look and feel of an app.

The default behavior with Windows Phone 8 was the inherent Silverlight behavior of scaling the image to fit. This meant that unless a high-resolution image was used, the image would end up looking blurred. Windows Runtime takes a different approach.

Windows 8x supports multitudes of screen sizes, resolutions and DPI. Windows Phone now supports 4 resolutions. The way Windows Runtime deals with this is by allowing developer to specify multiple images to match certain scale factors.

Windows Store Apps Windows Phone Store Apps
1.0 (100% no scaling) 1.0 (100% no scaling)
1.4 (140% scaling) 1.4 (140% scaling)
1.8 (180% scaling) 2.4 (240% scaling)

The runtime would determine the scaling factor to use depending upon the screen size, resolution, DPI and the form factor of device in question. The developer is required to supply images for scaling 1; the other scale factors are optional. If they are supplied, they will be used if necessary.

There are two ways of supplying images to support multi-scaling support.

  • Specify image scale within the file name e.g. Image.scale-100.png or Image.scale-140.png
  • Create direct for each image scale and place images inside without specifying scaling like in 1. E.g. scale-100/Image.png or scale-140/Image.png

The attached project includes examples of both ways.

To use this and enable auto-scaling support, you specify the image as specified below

<Image Source="Assets/Iris.png" />
<Image Source="Assets/Iris2.png" />

Occasionally it is necessary to set the images from code. In those instances, one should query the windows runtime to determine the current scaling factor and return appropriate image resource.

var uri = new System.Uri("ms-appx:///Assets/Iris.png");
var file = await StorageFile.GetFileFromApplicationUriAsync(uri);

This will correctly detect the scaling factor and get appropriate file.

Alternatively, for Windows 8x, we can use DisplayProperties.ResolutionScale to determine current scaling and for Windows Phone, we can use DisplayInformation.RawPixelsPerViewPixel.

Uri uri = null;
#if WINDOWS_PHONE_APP
    var rawpixelperview = Windows.Graphics.Display.DisplayInformation.GetForCurrentView().RawPixelsPerViewPixel; 

    var Width = Convert.ToInt32(Window.Current.Bounds.Width * rawpixelperview); 
    var Height = Convert.ToInt32(Window.Current.Bounds.Height * rawpixelperview); 

    if(Width == 480 && Height == 800)
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-100/Iris2.png");
    }
    else if((Width == 720 && Height == 1280 ) || (Width == 768 && Height == 1280))
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-140/Iris2.png");
    }
    else
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-240/Iris2.png");
    }
#else
    switch (DisplayProperties.ResolutionScale)
    {
        case ResolutionScale.Scale100Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-100/Iris2.png");
            break;

        case ResolutionScale.Scale140Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-140/Iris2.png");
            break;

        case ResolutionScale.Scale180Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-180/Iris2.png");
            break;
    }
#endif

var file = await StorageFile.GetFileFromApplicationUriAsync(uri);

It worth noting that images set in xaml and those done codebehind behave differently at least as far as size on screen is concerned. The only way to achieve correct size as visible in xaml is to set size to that of scale 1.0 image.

Supporting Accessibility

Windows Store apps support high contrast mode. The two available options are

  • Black background and white foreground
  • White background and black foreground.

To additionally support these two accessible image formats, you need to provide scale factor 1 images named / placed appropriately. I have excluded the high contrast images from sample as windows phone was using that by default.

Assets/Iris.scale-100_contrast-black.png

Assets/Iris.scale-100_contrast-white.png

or

Assets/scale-100_contrast-black/Iris2.png

Assets/scale-100_contrast-white/Iris2.png

Downloadable sample code can be downloaded here

App tile transparency for WP8.1 apps #wpdev

One of my popular lines is “Don’t throw shit around.. what goes around comes around!” and rightly so. A few weeks back I was making fun of @dvlup for blogging about how to create transparent tiles for the WP apps 🙂

I know the irony isn’t lost on me. The idea is with WP8 apps, you create tile images which are transparent png with white bits.. if you had a star or a clock that would be white, the rest would be transparent and you used this. This would make your app’s tile transparent and if you had a fancy background, you would see through the tile.

Fast forward today, I was testing WP8.1 update to my alarm clock app. What does it do ? well shows you tile update every minute with a nicely rendered tile. and yes I made the tile transparent. But the tile comes up as grey.. WTH

alarmclock1

so I start looking around and I remember that Windows 8 app manifest had a background color, Open the Package.appmanifest and I find this

<m3:VisualElements
    DisplayName="AlarmClock"
    Square150x150Logo="Assets\SquareTile150x150.png"
    Square44x44Logo="Assets\Logo.png"
    Description="AlarmClock"
    ForegroundText="light"
    BackgroundColor="#464646">

hmmm no alpha value to make it transparent. So I tried to set BackgroundColor to Transparent.. nope.. transparent ? Bingo!!

<m3:VisualElements
    DisplayName="AlarmClock"
    Square150x150Logo="Assets\SquareTile150x150.png"
    Square44x44Logo="Assets\Logo.png"
    Description="AlarmClock"
    ForegroundText="light"
    BackgroundColor="transparent">

alarmclock2
Now if you used transparent png for tiles, your WP 8.1 app’s tiles are transparent.

Image scaling related crashes on 1080p devices #wpdev

Back in March one of the 1Shot camera app users emailed me stating that the app crashes when using he uses digital zoom on Lumia 1520. I couldn’t reproduce the issue on my 1020. I then borrowed a friend’s 1520 and hmm sure I could reproduce it but I couldn’t catch the exception.

So we left it at that.. fast forward on Friday I get a mail back from chap in question and he states that he’s running latest greatest version of preview and the freaky crash is still happening. Luckily I have the 1520 at my desk and decided to put it to good use over the weekend.

So I put VS configuration in Debug mode and I set VS to catch all thrown exceptions. Run app and swipe left right and top bottom and every so often the app crashes.. no exception being caught.. the app just disappears and we are back to home screen. The only clue was the output of debug mode.. The program ‘[4880] AgHost.exe’ has exited with code -1073741819 (0xc0000005) ‘Access violation’.

The sad reality, the debug output is littered with messages starting ‘AgHost.exe’ (CoreCLR: Silverlight AppDomain): A quick check on the net showed that AgHost.exe is used to manage access through identity verification.. great. The error code wasn’t particularly useful either.

In this particular case, the error only occurred when zooming in. My implementation of digital zoom was scaling the preview image for the live view. The only action associated with crash was zooming in and out. As it happens, I detect manipulation delta and compute scale..

private void LivePreviewTapTarget_ManipulationDelta(object sender, System.Windows.Input.ManipulationDeltaEventArgs e)
{
    if (!this.viewModel.CaptureInitiated && AppSettings.InternalAppSettings.DragZoom)
    {
        SetDragToScale(e.CumulativeManipulation.Translation);
        e.Handled = true;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

My initial thought was that it being a newer SoC, the Xaml Renderer was somehow being proactive somewhere and getting itself in a mess.

Test 1: show less frames and show larger jumps so there is less rendering. For High Res camera I used constant of 0.01.. increase it to 0.05.

The resultant build crashed very very fast. hardly took a swipe or two to head back to home screen.

So I wanted to try the reverse but by laptop decided to shutdown and I was too knackered to continue yesterday.
Earlier today I had a chat with Bill Reiss.. he mentioned that he had some problems with 1520 too and that it had something to do with how large the image becomes on 1080p screen.

So I decided to continue where I left yesterday but with a reduction only for 1080p devices.
Test 2: reduce the multiplication constant to 0.0015 if a 1080p device is detected.
Result: Significant reduction in crashes.. I can still get it to crash but not that often. Here is the changed version

int DeviceScaleFactor = -1;
private void GetScaleFactor()
{
    object physicalScreenResolutionObject;

    if (DeviceExtendedProperties.TryGetValue("PhysicalScreenResolution", out physicalScreenResolutionObject))
    {
        var physicalScreenResolution = (Size)physicalScreenResolutionObject;

        DeviceScaleFactor = (int)(physicalScreenResolution.Width / 4.8);
    }
    else
    {
        DeviceScaleFactor = Application.Current.Host.Content.ScaleFactor;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    if (this.DeviceScaleFactor == -1)
    {
        this.GetScaleFactor();
    }

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (DeviceScaleFactor == 225)
    {
        scale = 0.0015 * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));
    }

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

if the Device scale factor is 225, its a 1080p device and one should exercise caution.

Certification Requirements update: Location Services Access #wpdev

Every since I can remember (at least 2 years now), the windows phone store’s certification requirements required:

  1. A store link to privacy policy
  2. Option to disable app’s use of location services.

If you didn’t do either of those two, the certification would fail (after about 7 days) and you would start again. However a fortnight back my Cineworld app failed a tiny update why ??? cause the update to certification requirements dictate that you also provide in app privacy policy (or a link) – not just on store.

Here are the new set of requirements:

App Policies: 2.7.2 Location Services Access

The privacy policy of your app must inform users about how location data from the Location Service API is used and disclose and the controls that users have over the use and sharing of location data. This can be hosted within or directly linked from the app. The privacy policy must be accessible from your app at any time.

Notification/Action Requested:

The app uses location services, however one or more of the following is missing:

  • A persistent privacy policy in-app
  • A privacy policy URL that is accessible through the Windows Phone Store
  • A privacy policy that is unique to the app

If you encounter certification failure, add a hyperlink button and point it to online privacy policy at the very least. Oh and in case you are wondering,  this used to be a Window 8 Store requirement – one store coming to devices near you ???

Photo / Camera API – 1 #wpdev

My memory of Windows Phone NoDo API set is pretty bleak. I distinctly remember presence of CameraCaptureTask and PhotoChooserTask. I think I used PhotoChooserTask with my first app Scribble. So what’s this post about ? well Photo / Camera options available on Windows Phone 8.

Since I started with the two tasks, they still exist in Windows Phone 8.

CameraCaptureTask: this fires the stock camera app – ability to navigate to camera roll and use other lens is blocked but otherwise it is the stock windows phone app. Tap to focus / capture or use dedicated shutter button. On capture you are presented with a preview and you have the option to accept or retake the capture.

wp_ss_20140130_0005 wp_ss_20140130_0006 wp_ss_20140130_0007

PhotoChooserTask: this is a different beast. This opens photos hub and presents user with list of albums allowing user to select a photo. The Task has a parameter that toggles visibility of camera icon – ability to not just select a picture but also initiate CameraCaptureTask from within.

While both of these are very basic, let me point out that Instagram BETA still uses a layout very similar to Photochooser task and obviously uses Camera Capture Task.

Its very likely that Instagram could get away with murder. Most of us – 3rd party developers however do not have such luxury. We would be termed as lazy if not worse. And you might not want the stock camera UI. So what can you do ? I am still in old API mode so let me mention an API that was part of Mango – I don’t know if NoDo had it or not – I only used it after Mango public release in my Cool Camera app.

PhotoCamera Class: this class provides raw access to camera and exposes a few options like resolution, flash and focus. Apps render live view by means of using a video brush whose source is set to an instance of photo camera class. How to create a base camera app for Windows Phone provides a great sample to kick start camera app development using this class

PhotoCamera

For a camera app, call CaptureImage. Others might not need a full blown capture – something like QR scanning app would just use focus and can do with GetPreviewBufferArgb32.

Whilst this is a good start, the API is not extensive enough for anything beyond basic. In the next post, I will discuss PhotoCaptureDevice class. This class though simple, exposes enough API to create a full blown camera app allowing you to control every single aspect of camera. If you hardware supports it, you should be able to utilise it

Camera / Imaging fragmentation #wp8 #wpdev #windowsphone

This post has been long coming… should be started it back when I first heard of Nokia Music API prior to new of Microsoft’s purchase of Nokia Devices division.

We all know Microsoft Windows Phone Dev team isn’t the fastest beast when it comes to pushing out new API. Many (myself included) welcomed Nokia’s SDKs with open arms as they can turn things around much faster.

We know that public 3rd Party Windows Phone API is fixed and involves querying for supported values – this makes supporting various bits of functionality across manufacturers easy. I am not referring to fragmentation of this.

When Nokia announced 1020, they pushed for a concept of reframing. They take two pics, high resolution and a low resolution. you as a user can only see low res on device. It doesn’t just end there, even the OS itself even through it can handle high res images is only aware of low res images. Wait!! What are am I going on about ? The OS doesn’t know about high res images even through they physically reside on the same physical directory.

If you ever wondered about how the images are stored.. let me give you a clue. according to Windows Phone, the camera roll points to “C:\Data\Users\Public\Pictures\Camera Roll” something the shared Windows NT kernel gave WP8 🙂

If you try to programmatically access this location, you are greeted with System.UnauthorizedAccessException.

With Lumia Black update, Nokia is bringing DNG support to devices. Whilst this is good news on the surface, it uses the same hack as the one used for high res – the device is unaware of it.

This leads to multitudes of device tiers hardware and software capabilities which are carefully controlled outside of public API and outside of Microsoft WPDev team itself.

At some point last year, I did ask Stephen Elop if APIs will merge once Devices division merges.. sadly all he replied was that he couldn’t comment on future.

1Shot camera app #WP8 #WindowsPhone

It started off with Slydr, then Cineworld. Now I have another app that takes up most of my coding time. 1Shot camera app.

1Shot beta app is based on my usage patterns. If you like Nokia Camera (and its not bad), I am happy for you. If however like me, you wish that you could see High Resolution images on camera roll or you just don’t like mucking around with manual settings rather often choose modes, then 1Shot is the app for you.

By default you get on screen options to tweak Flash modes, use Timer, use multiple capture mode and additionally use camera modes exposed by device. Today I submitted a change that allows use to turn on Advanced camera options to view ISO, Shutter Speed, Exposure Compensation and White Balance Presets.

The beauty of 1Shot app is

  • its very simple interface
  • you can use it on any WP8 device.
  • If your device supports High resolution capture like 1520 or 1020, the app will use those and copy the high res image straight to camera roll.
  • All images captures are made available for skydrive sync.

There’s still a lot more that needs to be done. You can download the app from Windows Phone Store – just remember, its still early days and is labelled Beta for that reason

http://www.windowsphone.com/s?appid=4b823a7e-5316-4117-877e-006fcc4c6083