Images and Scaling with Windows Runtime #WinRT #WPDev #Windev

With the advent of Windows Phone 8.1, developers have the option of using WinRT native XAML stack as opposed to the Silverlight XAML the only option with early Windows Phone iterations.

One of the differences between WinRT XAML and Silverlight XAML is scaling and like with fonts, & images, scaling can have a significant effect on the look and feel of an app.

The default behavior with Windows Phone 8 was the inherent Silverlight behavior of scaling the image to fit. This meant that unless a high-resolution image was used, the image would end up looking blurred. Windows Runtime takes a different approach.

Windows 8x supports multitudes of screen sizes, resolutions and DPI. Windows Phone now supports 4 resolutions. The way Windows Runtime deals with this is by allowing developer to specify multiple images to match certain scale factors.

Windows Store Apps Windows Phone Store Apps
1.0 (100% no scaling) 1.0 (100% no scaling)
1.4 (140% scaling) 1.4 (140% scaling)
1.8 (180% scaling) 2.4 (240% scaling)

The runtime would determine the scaling factor to use depending upon the screen size, resolution, DPI and the form factor of device in question. The developer is required to supply images for scaling 1; the other scale factors are optional. If they are supplied, they will be used if necessary.

There are two ways of supplying images to support multi-scaling support.

  • Specify image scale within the file name e.g. Image.scale-100.png or Image.scale-140.png
  • Create direct for each image scale and place images inside without specifying scaling like in 1. E.g. scale-100/Image.png or scale-140/Image.png

The attached project includes examples of both ways.

To use this and enable auto-scaling support, you specify the image as specified below

<Image Source="Assets/Iris.png" />
<Image Source="Assets/Iris2.png" />

Occasionally it is necessary to set the images from code. In those instances, one should query the windows runtime to determine the current scaling factor and return appropriate image resource.

var uri = new System.Uri("ms-appx:///Assets/Iris.png");
var file = await StorageFile.GetFileFromApplicationUriAsync(uri);

This will correctly detect the scaling factor and get appropriate file.

Alternatively, for Windows 8x, we can use DisplayProperties.ResolutionScale to determine current scaling and for Windows Phone, we can use DisplayInformation.RawPixelsPerViewPixel.

Uri uri = null;
#if WINDOWS_PHONE_APP
    var rawpixelperview = Windows.Graphics.Display.DisplayInformation.GetForCurrentView().RawPixelsPerViewPixel; 

    var Width = Convert.ToInt32(Window.Current.Bounds.Width * rawpixelperview); 
    var Height = Convert.ToInt32(Window.Current.Bounds.Height * rawpixelperview); 

    if(Width == 480 && Height == 800)
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-100/Iris2.png");
    }
    else if((Width == 720 && Height == 1280 ) || (Width == 768 && Height == 1280))
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-140/Iris2.png");
    }
    else
    {
        uri2 = new System.Uri("ms-appx:///Assets/scale-240/Iris2.png");
    }
#else
    switch (DisplayProperties.ResolutionScale)
    {
        case ResolutionScale.Scale100Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-100/Iris2.png");
            break;

        case ResolutionScale.Scale140Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-140/Iris2.png");
            break;

        case ResolutionScale.Scale180Percent:
            uri2 = new System.Uri("ms-appx:///Assets/scale-180/Iris2.png");
            break;
    }
#endif

var file = await StorageFile.GetFileFromApplicationUriAsync(uri);

It worth noting that images set in xaml and those done codebehind behave differently at least as far as size on screen is concerned. The only way to achieve correct size as visible in xaml is to set size to that of scale 1.0 image.

Supporting Accessibility

Windows Store apps support high contrast mode. The two available options are

  • Black background and white foreground
  • White background and black foreground.

To additionally support these two accessible image formats, you need to provide scale factor 1 images named / placed appropriately. I have excluded the high contrast images from sample as windows phone was using that by default.

Assets/Iris.scale-100_contrast-black.png

Assets/Iris.scale-100_contrast-white.png

or

Assets/scale-100_contrast-black/Iris2.png

Assets/scale-100_contrast-white/Iris2.png

Downloadable sample code can be downloaded here

WinRT License API, Silverlight 8.1 apps and sideload detection #wpdev

Since early 2011, I have been using the method described here.
Recently I have been upgrading all my apps to target Windows Phone 8.1 API. Most of previous API is still accessible in WP 8.1 SL8.1 apps. However a great deal more is available under WinRT API.

With my Alarm Clock app, I have started using WinRT API to pin and update Tiles. I have used WinRT API for background tasks as I can run them more frequently and I can detect Timezone change. I have added IAP to the app.
The app however is available in 2 flavours Free (with ads and restrictions) and Paid. So far I have only been updating the paid app. Of course I use sideload detection for Paid app however while considering update of Free version, I decided to use CurrenApp to execute License check and potentially offer to convert Free users in-place as opposed to getting them to buy another app!!

In good old SL API, I would have used

var license = new Microsoft.Phone.Marketplace.LicenseInformation();
IsTrial = license.IsTrial();

With WinRT that becomes

IsTrial = CurrentApp.LicenseInformation.IsTrial;

While developing and deploying if you observe the behaviour, the IsTrial never returns false. You cannot test Trial scenario without explicit pre-processor directives!
Of course CurrentAppSimulator is only available for WinRT app and not SL8.1 apps. The same behaviour would be exposed by the sideloaded apps. Not downloaded from store equates to full version. Like in good old days, WMAppPRHeader.xml is still attached after certification. So lets modify the code

try
{
   XDocument doc = XDocument.Load("WMAppPRHeader.xml");
 
   IsTrial = CurrentApp.LicenseInformation.IsTrial;
}
catch
{
   IsTrial = true;
}

You treat all apps without WMAppPRHeader.xml as trial. Job done!
Note that IAP are not affected. If you had TestIAP, it is not active by default.

CurrentApp.LicenseInformation.ProductLicenses[&amp;quot;TestIAP&amp;quot;].IsActive;

Happy coding!

DLL / Reference hell with #WP8 #SL81 and #WinRT component

Over the last month, I have gone back to “AlarmClock” app (my 3rd Windows Phone app from back in 2011). I have updated it to render time updates on live tile. In fact I am quite proud the mechanism I use. I additionally provided 50 downloadable font packs that users can choose from (purchasable in a set of 10).

Original solution
The update mechanism was initially based on Silverlight Background Task. The tile update mechanism of course updates tile based on local time. However a user reported that when travelling between timezones, the tiles kept showing previous timezone times until the next background task run started pushing new ones based on new local time.

I deliberated on this and came to the conclusion that somewhere something was fishy.. its possible that Tile updates are actually scheduled on UTC as opposed to local time or maybe I just don’t understand it. Either way, I had to fix this issue

Solution 2: Fix for Timezone change
The new windows phone 8.1 introduced a bunch of new WinRT API that included API for Background task including those that run on Timezone change. So I dumped SL based Background Task and went full stream with WinRT component. Added 2 WinRT classs on for Timer and another for Timezone and this worked great.

Another user wanted the clock and tile updates to support multiple Timezones – wanted to pin Hong Kong and Seattle. My app so far only supported local device timezone.

Soluion 3 – Multiple Timezone support

.NET 4.5 comes with TimeZoneInfo class.This provides a nice method GetSystemTimeZones which lists all timezones and make it possible to convert between those. However Windows Phone .NET Runtime does not come with this and there is not suitable WinRT API either. Yay!!
Oren Novotny created a good Win32 wrapper which works on both WP8, WP81, SL81 and WinRT
I fired up Nuget Package Manager and searched for timezone..found Timezone conversion between WinRT and PCL.. Added a reference to the App and reworked it to support user selectable timezone etc. This required me to select target between x86 and ARM – any CPU no longer an option.

Next step?? Add Timezone helper reference to WinRT component. Bang bang bang.. nugget sorted it all out.. that was easy.. modified the background processor to update all tiles with correct timezone times!! nice :) What could possibly go wrong ?

Hit F6, (Old school.. modified VS 2013 to build o F6 like before).. all good. Hit F5… wait for it !!!!

Runtime exception – File not found System.Runtime.InteropService.. quick search showed reference conflict..

Step 1) Set WinRT component to x86 instead of Any CPU.. maybe that will do the trick!!!

Nope far from it… Result System.BadImageFormatException thown when trying to run the app.

Step 2) Download code and make a part of existing WinRT component..

Nope still the same error(s)

While deliberating, realised that I called code from Background Task directly from my app.. 3rd trial

Step 3)

Remove calls to code in Background Task from App
Nuget Reference to App
Code within WinRT component for internal use
App – x86 / ARM target
WinRT component – Any CPU tagt

Bingo.. It all work as advertised!!

Took me day to figure try out various options

Image scaling related crashes on 1080p devices #wpdev

Back in March one of the 1Shot camera app users emailed me stating that the app crashes when using he uses digital zoom on Lumia 1520. I couldn’t reproduce the issue on my 1020. I then borrowed a friend’s 1520 and hmm sure I could reproduce it but I couldn’t catch the exception.

So we left it at that.. fast forward on Friday I get a mail back from chap in question and he states that he’s running latest greatest version of preview and the freaky crash is still happening. Luckily I have the 1520 at my desk and decided to put it to good use over the weekend.

So I put VS configuration in Debug mode and I set VS to catch all thrown exceptions. Run app and swipe left right and top bottom and every so often the app crashes.. no exception being caught.. the app just disappears and we are back to home screen. The only clue was the output of debug mode.. The program ‘[4880] AgHost.exe’ has exited with code -1073741819 (0xc0000005) ‘Access violation’.

The sad reality, the debug output is littered with messages starting ‘AgHost.exe’ (CoreCLR: Silverlight AppDomain): A quick check on the net showed that AgHost.exe is used to manage access through identity verification.. great. The error code wasn’t particularly useful either.

In this particular case, the error only occurred when zooming in. My implementation of digital zoom was scaling the preview image for the live view. The only action associated with crash was zooming in and out. As it happens, I detect manipulation delta and compute scale..

private void LivePreviewTapTarget_ManipulationDelta(object sender, System.Windows.Input.ManipulationDeltaEventArgs e)
{
    if (!this.viewModel.CaptureInitiated && AppSettings.InternalAppSettings.DragZoom)
    {
        SetDragToScale(e.CumulativeManipulation.Translation);
        e.Handled = true;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

My initial thought was that it being a newer SoC, the Xaml Renderer was somehow being proactive somewhere and getting itself in a mess.

Test 1: show less frames and show larger jumps so there is less rendering. For High Res camera I used constant of 0.01.. increase it to 0.05.

The resultant build crashed very very fast. hardly took a swipe or two to head back to home screen.

So I wanted to try the reverse but by laptop decided to shutdown and I was too knackered to continue yesterday.
Earlier today I had a chat with Bill Reiss.. he mentioned that he had some problems with 1520 too and that it had something to do with how large the image becomes on 1080p screen.

So I decided to continue where I left yesterday but with a reduction only for 1080p devices.
Test 2: reduce the multiplication constant to 0.0015 if a 1080p device is detected.
Result: Significant reduction in crashes.. I can still get it to crash but not that often. Here is the changed version

int DeviceScaleFactor = -1;
private void GetScaleFactor()
{
    object physicalScreenResolutionObject;

    if (DeviceExtendedProperties.TryGetValue("PhysicalScreenResolution", out physicalScreenResolutionObject))
    {
        var physicalScreenResolution = (Size)physicalScreenResolutionObject;

        DeviceScaleFactor = (int)(physicalScreenResolution.Width / 4.8);
    }
    else
    {
        DeviceScaleFactor = Application.Current.Host.Content.ScaleFactor;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    if (this.DeviceScaleFactor == -1)
    {
        this.GetScaleFactor();
    }

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (DeviceScaleFactor == 225)
    {
        scale = 0.0015 * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));
    }

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

if the Device scale factor is 225, its a 1080p device and one should exercise caution.

Where did the SystemTray vanish with #WinPRT #wpdev?

With Silverlight WP8 API, SystemTray is exposed in PhoneApplicationPage through Microsoft.Phone.Shell.

<phone:PhoneApplicationPage
    shell:SystemTray.IsVisible="True" 
    shell:SystemTray.BackgroundColor="AliceBlue"
    shell:SystemTray.ForegroundColor="Black"
    shell:SystemTray.Opacity="0.75">
    <shell:SystemTray.ProgressIndicator>
        <shell:ProgressIndicator />
    </shell:SystemTray.ProgressIndicator>
</phone:PhoneApplicationPage>
SystemTray.BackgroundColor = Colors.Cyan;
SystemTray.ForegroundColor = Colors.White;
SystemTray.IsVisible = true;
SystemTray.Opacity = 0.8;
SystemTray.ProgressIndicator = new ProgressIndicator() { IsIndeterminate = true, IsVisible = true };

it was very easily done through xaml itself. You could do it from codebehind by accessing static properties of SystemTray class.
However with WinPRT, the Page class does not expose SystemTray and now it lives as Windows.UI.ViewManagement.StatusBar.
No xaml declarative business here.

var statusBar = StatusBar.GetForCurrentView();
statusBar.BackgroundColor = Colors.Black;
statusBar.BackgroundOpacity = 0.8;
statusBar.ForegroundColor = Colors.White;
statusBar.ProgressIndicator.Text = "loading";
await statusBar.ShowAsync();

If I find anything declarative, I will post those at a later date.

Backbutton handling with #winprt and Windows Phone 8.1 #wpdev

Back with WP8 and Silverlight API, handling hardware back button was easy peasy. By default, PhoneApplicationPage exposed an eventhandler called BackKeyPress. Developers could chose to subscribe to this event handler and handle the back button.

<phone:PhoneApplicationPage shell:SystemTray.IsVisible="True" BackKeyPress="PhoneApplicationPage_BackKeyPress">
...
...
...
</phone:PhoneApplicationPage>
private void PhoneApplicationPage_BackKeyPress(object sender, System.ComponentModel.CancelEventArgs e)
{
    if (this.DrawingCanvas.Visibility == System.Windows.Visibility.Visible)
    {
        this.DrawingCanvas.Visibility = this.ColorSlider.Visibility = System.Windows.Visibility.Collapsed;
        this.piWords.IsLocked = false;
        e.Cancel = true;
    }
}

As you can see, in the back key press, I check to see if certain view is being shown and if so, I hide it that rather than navigating page backstack.

That was Silverlight / WP API.. how would we do that in WP8.1 / WinPRT API ?
Windows.Phone.UI.Input namespace contains a class called HardwareButtons. just remember this is WP8.1 WinPRT API only – not supported on Windows store apps. Sadly no XAML wiring up.. lets go to codebehind and subscribe to it


public MainPage()
{
    this.InitializeComponent();

    this.NavigationCacheMode = NavigationCacheMode.Required;

    HardwareButtons.BackPressed += HardwareButtons_BackPressed;
}

void HardwareButtons_BackPressed(object sender, BackPressedEventArgs e)
{
    if (some_condition)
    {
        e.Handled = true;
    }
}

That’s about it.. thank goodness I still have Dev preview documentation lying around :)

Photo / Camera API – 1 #wpdev

My memory of Windows Phone NoDo API set is pretty bleak. I distinctly remember presence of CameraCaptureTask and PhotoChooserTask. I think I used PhotoChooserTask with my first app Scribble. So what’s this post about ? well Photo / Camera options available on Windows Phone 8.

Since I started with the two tasks, they still exist in Windows Phone 8.

CameraCaptureTask: this fires the stock camera app – ability to navigate to camera roll and use other lens is blocked but otherwise it is the stock windows phone app. Tap to focus / capture or use dedicated shutter button. On capture you are presented with a preview and you have the option to accept or retake the capture.

wp_ss_20140130_0005 wp_ss_20140130_0006 wp_ss_20140130_0007

PhotoChooserTask: this is a different beast. This opens photos hub and presents user with list of albums allowing user to select a photo. The Task has a parameter that toggles visibility of camera icon – ability to not just select a picture but also initiate CameraCaptureTask from within.

While both of these are very basic, let me point out that Instagram BETA still uses a layout very similar to Photochooser task and obviously uses Camera Capture Task.

Its very likely that Instagram could get away with murder. Most of us – 3rd party developers however do not have such luxury. We would be termed as lazy if not worse. And you might not want the stock camera UI. So what can you do ? I am still in old API mode so let me mention an API that was part of Mango – I don’t know if NoDo had it or not – I only used it after Mango public release in my Cool Camera app.

PhotoCamera Class: this class provides raw access to camera and exposes a few options like resolution, flash and focus. Apps render live view by means of using a video brush whose source is set to an instance of photo camera class. How to create a base camera app for Windows Phone provides a great sample to kick start camera app development using this class

PhotoCamera

For a camera app, call CaptureImage. Others might not need a full blown capture – something like QR scanning app would just use focus and can do with GetPreviewBufferArgb32.

Whilst this is a good start, the API is not extensive enough for anything beyond basic. In the next post, I will discuss PhotoCaptureDevice class. This class though simple, exposes enough API to create a full blown camera app allowing you to control every single aspect of camera. If you hardware supports it, you should be able to utilise it