Image scaling related crashes on 1080p devices #wpdev

Back in March one of the 1Shot camera app users emailed me stating that the app crashes when using he uses digital zoom on Lumia 1520. I couldn’t reproduce the issue on my 1020. I then borrowed a friend’s 1520 and hmm sure I could reproduce it but I couldn’t catch the exception.

So we left it at that.. fast forward on Friday I get a mail back from chap in question and he states that he’s running latest greatest version of preview and the freaky crash is still happening. Luckily I have the 1520 at my desk and decided to put it to good use over the weekend.

So I put VS configuration in Debug mode and I set VS to catch all thrown exceptions. Run app and swipe left right and top bottom and every so often the app crashes.. no exception being caught.. the app just disappears and we are back to home screen. The only clue was the output of debug mode.. The program ‘[4880] AgHost.exe’ has exited with code -1073741819 (0xc0000005) ‘Access violation’.

The sad reality, the debug output is littered with messages starting ‘AgHost.exe’ (CoreCLR: Silverlight AppDomain): A quick check on the net showed that AgHost.exe is used to manage access through identity verification.. great. The error code wasn’t particularly useful either.

In this particular case, the error only occurred when zooming in. My implementation of digital zoom was scaling the preview image for the live view. The only action associated with crash was zooming in and out. As it happens, I detect manipulation delta and compute scale..

private void LivePreviewTapTarget_ManipulationDelta(object sender, System.Windows.Input.ManipulationDeltaEventArgs e)
{
    if (!this.viewModel.CaptureInitiated && AppSettings.InternalAppSettings.DragZoom)
    {
        SetDragToScale(e.CumulativeManipulation.Translation);
        e.Handled = true;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

My initial thought was that it being a newer SoC, the Xaml Renderer was somehow being proactive somewhere and getting itself in a mess.

Test 1: show less frames and show larger jumps so there is less rendering. For High Res camera I used constant of 0.01.. increase it to 0.05.

The resultant build crashed very very fast. hardly took a swipe or two to head back to home screen.

So I wanted to try the reverse but by laptop decided to shutdown and I was too knackered to continue yesterday.
Earlier today I had a chat with Bill Reiss.. he mentioned that he had some problems with 1520 too and that it had something to do with how large the image becomes on 1080p screen.

So I decided to continue where I left yesterday but with a reduction only for 1080p devices.
Test 2: reduce the multiplication constant to 0.0015 if a 1080p device is detected.
Result: Significant reduction in crashes.. I can still get it to crash but not that often. Here is the changed version

int DeviceScaleFactor = -1;
private void GetScaleFactor()
{
    object physicalScreenResolutionObject;

    if (DeviceExtendedProperties.TryGetValue("PhysicalScreenResolution", out physicalScreenResolutionObject))
    {
        var physicalScreenResolution = (Size)physicalScreenResolutionObject;

        DeviceScaleFactor = (int)(physicalScreenResolution.Width / 4.8);
    }
    else
    {
        DeviceScaleFactor = Application.Current.Host.Content.ScaleFactor;
    }
}

void SetDragToScale(System.Windows.Point pScale)
{
    double x = Math.Abs(pScale.X);
    double y = Math.Abs(pScale.Y);

    double manipulation = x > y ? pScale.X : pScale.Y;

    bool zoomOut = manipulation >= 0 ? false : true;

    if (this.DeviceScaleFactor == -1)
    {
        this.GetScaleFactor();
    }

    double scale = (this.viewModel.HasHighResolutionCamera ? 0.01 : 0.005) * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));

    if (DeviceScaleFactor == 225)
    {
        scale = 0.0015 * Math.Sqrt(Math.Pow(x, 2) + Math.Pow(y, 2));
    }

    if (zoomOut)
        this.viewModel.ViewScale = initialScale + scale;
    else
        this.viewModel.ViewScale = initialScale - scale;

    System.Diagnostics.Debug.WriteLine("zoomout {0}, scale {1}", zoomOut, scale);
}

if the Device scale factor is 225, its a 1080p device and one should exercise caution.

Where did the SystemTray vanish with #WinPRT #wpdev?

With Silverlight WP8 API, SystemTray is exposed in PhoneApplicationPage through Microsoft.Phone.Shell.

<phone:PhoneApplicationPage
    shell:SystemTray.IsVisible="True" 
    shell:SystemTray.BackgroundColor="AliceBlue"
    shell:SystemTray.ForegroundColor="Black"
    shell:SystemTray.Opacity="0.75">
    <shell:SystemTray.ProgressIndicator>
        <shell:ProgressIndicator />
    </shell:SystemTray.ProgressIndicator>
</phone:PhoneApplicationPage>
SystemTray.BackgroundColor = Colors.Cyan;
SystemTray.ForegroundColor = Colors.White;
SystemTray.IsVisible = true;
SystemTray.Opacity = 0.8;
SystemTray.ProgressIndicator = new ProgressIndicator() { IsIndeterminate = true, IsVisible = true };

it was very easily done through xaml itself. You could do it from codebehind by accessing static properties of SystemTray class.
However with WinPRT, the Page class does not expose SystemTray and now it lives as Windows.UI.ViewManagement.StatusBar.
No xaml declarative business here.

var statusBar = StatusBar.GetForCurrentView();
statusBar.BackgroundColor = Colors.Black;
statusBar.BackgroundOpacity = 0.8;
statusBar.ForegroundColor = Colors.White;
statusBar.ProgressIndicator.Text = "loading";
await statusBar.ShowAsync();

If I find anything declarative, I will post those at a later date.

Backbutton handling with #winprt and Windows Phone 8.1 #wpdev

Back with WP8 and Silverlight API, handling hardware back button was easy peasy. By default, PhoneApplicationPage exposed an eventhandler called BackKeyPress. Developers could chose to subscribe to this event handler and handle the back button.

<phone:PhoneApplicationPage shell:SystemTray.IsVisible="True" BackKeyPress="PhoneApplicationPage_BackKeyPress">
...
...
...
</phone:PhoneApplicationPage>
private void PhoneApplicationPage_BackKeyPress(object sender, System.ComponentModel.CancelEventArgs e)
{
    if (this.DrawingCanvas.Visibility == System.Windows.Visibility.Visible)
    {
        this.DrawingCanvas.Visibility = this.ColorSlider.Visibility = System.Windows.Visibility.Collapsed;
        this.piWords.IsLocked = false;
        e.Cancel = true;
    }
}

As you can see, in the back key press, I check to see if certain view is being shown and if so, I hide it that rather than navigating page backstack.

That was Silverlight / WP API.. how would we do that in WP8.1 / WinPRT API ?
Windows.Phone.UI.Input namespace contains a class called HardwareButtons. just remember this is WP8.1 WinPRT API only – not supported on Windows store apps. Sadly no XAML wiring up.. lets go to codebehind and subscribe to it


public MainPage()
{
    this.InitializeComponent();

    this.NavigationCacheMode = NavigationCacheMode.Required;

    HardwareButtons.BackPressed += HardwareButtons_BackPressed;
}

void HardwareButtons_BackPressed(object sender, BackPressedEventArgs e)
{
    if (some_condition)
    {
        e.Handled = true;
    }
}

That’s about it.. thank goodness I still have Dev preview documentation lying around :)

Photo / Camera API – 1 #wpdev

My memory of Windows Phone NoDo API set is pretty bleak. I distinctly remember presence of CameraCaptureTask and PhotoChooserTask. I think I used PhotoChooserTask with my first app Scribble. So what’s this post about ? well Photo / Camera options available on Windows Phone 8.

Since I started with the two tasks, they still exist in Windows Phone 8.

CameraCaptureTask: this fires the stock camera app – ability to navigate to camera roll and use other lens is blocked but otherwise it is the stock windows phone app. Tap to focus / capture or use dedicated shutter button. On capture you are presented with a preview and you have the option to accept or retake the capture.

wp_ss_20140130_0005 wp_ss_20140130_0006 wp_ss_20140130_0007

PhotoChooserTask: this is a different beast. This opens photos hub and presents user with list of albums allowing user to select a photo. The Task has a parameter that toggles visibility of camera icon – ability to not just select a picture but also initiate CameraCaptureTask from within.

While both of these are very basic, let me point out that Instagram BETA still uses a layout very similar to Photochooser task and obviously uses Camera Capture Task.

Its very likely that Instagram could get away with murder. Most of us – 3rd party developers however do not have such luxury. We would be termed as lazy if not worse. And you might not want the stock camera UI. So what can you do ? I am still in old API mode so let me mention an API that was part of Mango – I don’t know if NoDo had it or not – I only used it after Mango public release in my Cool Camera app.

PhotoCamera Class: this class provides raw access to camera and exposes a few options like resolution, flash and focus. Apps render live view by means of using a video brush whose source is set to an instance of photo camera class. How to create a base camera app for Windows Phone provides a great sample to kick start camera app development using this class

PhotoCamera

For a camera app, call CaptureImage. Others might not need a full blown capture – something like QR scanning app would just use focus and can do with GetPreviewBufferArgb32.

Whilst this is a good start, the API is not extensive enough for anything beyond basic. In the next post, I will discuss PhotoCaptureDevice class. This class though simple, exposes enough API to create a full blown camera app allowing you to control every single aspect of camera. If you hardware supports it, you should be able to utilise it

Immutable Collections Performance #dotnet

Yesterday ayende blogged and tweeted about serious issues with Immutable collections performance. whilst I didn’t have time to really look into it, I did retweet the link and got involved in a discussion with ex-softie Brandon Paddock.

When I woke up, I saw some more discussion and thought, let me try it myself. Now we all know every chef has their own way and some ways are better than others. FYI, I never used ImmutableCollections before this date and it took me a few mins to get the hang of it.

so what did I do ? I kept it simple. Here’s the source and the results. As you can see creating immutable collection is an added option and that takes time but its not very high

class Program
{
    static int MaxIterations = 10 * 1000 * 1000;
    static int maxRuns = 5;
    static void Main(string[] args)
    {
        for (int i = 0; i < maxRuns; i++ )
        {
            GC.Collect();

            TestGenericList();

            GC.Collect();

            TestImmutableList();
        }
    }

    public static void PreRunWarmup()
    {
        List<int> genericDataList = new List<int>();

        for (int i = 0; i < MaxIterations; i++)
        {
            genericDataList.Add(i);
        }

        genericDataList.Clear();
        genericDataList = null;
    }

    public static void TestGenericList()
    {
        List<int> genericDataList = new List<int>();
                        
        var sw = Stopwatch.StartNew();
        for (int i = 0; i < MaxIterations; i++)
        {
            genericDataList.Add(i);
        }

        sw.Stop();
        Console.WriteLine("Generic List took {0} ms", (double)sw.ElapsedTicks * (double)1000 / (double)Stopwatch.Frequency);

        genericDataList.Clear();
        genericDataList = null;
    }

    public static void TestImmutableList()
    {
        List<int> genericDataList = new List<int>();

        var sw = Stopwatch.StartNew();
        for (int i = 0; i < MaxIterations; i++)
        {
            genericDataList.Add(i);
        }

        ImmutableList<List<int>> immutableList = ImmutableList.Create(genericDataList);

        sw.Stop();
        Console.WriteLine("Immutable List took {0} ms", (double)sw.ElapsedTicks * (double)1000 / (double)Stopwatch.Frequency);

        genericDataList.Clear();
        genericDataList = null;
    }
}

and the results are

Generic List took 85.5857446744927 ms
Immutable List took 88.5216447923091 ms
Generic List took 89.7324092764301 ms
Immutable List took 88.4739507537922 ms
Generic List took 88.1856734956681 ms
Immutable List took 90.4921934216639 ms
Generic List took 85.6533615392255 ms
Immutable List took 88.9282515637155 ms
Generic List took 87.9722577663554 ms
Immutable List took 84.6940472708294 ms
Press any key to continue . . .

Note: I added a pre-test warm up as the first iteration was taking unusually long.
So for a few more milli-seconds per million entries what do we get ?

If you have a threaded environment and you pass collection and iterate them, you should have come across the infamous InvalidOperationException stating something like “Collection was modified; enumeration operation may not execute.” At least I have resorted to using lock around collection enumeration code in past.
With immutable collection, you never have to worry about the output received from Immutable.CreateList. No more unnecessary locks around iterations.

Note: I used GC.Collect excessively and even resorted to cleaning and setting collections to null so as to not impact the next call. Just pure perf test.

Color Picker for #win8dev

I use colour picker across multiple projects on both windows phone and windows 8. Allowing users a choice of colour is a good thing from experience perspective. Whilst I found a number of solutions for Windows Phone, I never really found much for Windows 8.

Last year I ported the Coding4Fun ColorHexagonPicker to use with my TableClock app.Recenly working on Scribble I needed a broader range of colours and I decided to create an appbar control that conveniently lives in AppBar.

I have uploaded the source to Github. Feel free to fork and extent.

Screenshot (1)

 

Detect whether user has reviewed your app or not #wpdev

In May while trying to get Cineworld app into Nokia Collections, I was asked to up the ratings – it had about 143 3.5* ratings. Nokia contact gave me a feedback helper control that I plugged in and bang.. 100 reviews a week and today it stands at 1800+

the app still hasn’t made it to Nokia Collections but I has earned me some XP from DVLUP.

So what does the feedback helper do ? well it prompts the user to rate the app. Some times a user would have already rated the app and prompting them – ends up pissing them off. some down rate the app, others down rate and email me about it.

A while back I noticed that if you went to review an app that you had already reviewed, the app store would show you previous review and allow you to modify it. Encouraged by Pedro Lamas use of fiddler to detect updates, I fired up fiddler and gave it a whirls.. here are the findings.

GET https://marketplaceedgeservice.windowsphone.com/v8/ratings/product/9b255dee-4682-49d8-88bc-d568e40f84bc/user/reviews?os=8.0.10328.0&cc=GB&lang=en-GB&hw=520170499&dm=RM-821_eu_euro1_425 HTTP/1.1
Connection: Keep-Alive
Content-Type: application/xml
Accept: */*
Accept-Encoding: gzip
Authorization: WLID1.0 t=EwBwAsJkBAAUjOxKcvu5vfjrrkq8bAfVS/jJQx4AATQW7lCndkoL7NuO2uyyYc8Bizwpr0BSyTbldPkm+Ot9eyULF0ZNXgqseu3KMp5GX3UNiwaJuOkgpCHQb3n2mtHjIqOMLVQ8C5Mf+xNCiKT/uD9MH0oy8oT3FYMrIJzsqlX2XzaN6UgcuXHKCXWFKhc5pMGxc9STMzNVXIOJhWq17gi9oVCT7/d4qKmm9Zqn9domh0ahUHLGUxaoaBxLbhBTNzmg33ZiJzvZp7/HX4wDmG93OY1pdCC7vUw1ecJY9ZkxGguw+8uVE+cqO+Q411LUmPUA1XKLrxekJVNE2j9meZV89VF2N3ohfAUz/6b0DRZ0s1m0P8GM2U7MClRvfWYDZgAACN1ftULaeX7PQAECQSNrxGE7dsRpkTdnmFV5+S4fSKWdy7DeFTYFsPIxa/xaMQO4b/jG5SO5hGi/Apre2mMZfBYw3mSJUsJ9lraS9OVoIbNS6C9oDaPlhGdugQTI+V6DkHxB6uEQWZysFzTTMXOgoaoRxS9HbQKbRrmi0hA7gGAMHvEZ5DmXa9VY0Q5lEQnrbecNZe6utgaLvLRhFPolUBqP06HTVqiwmvIVZ6NgBa7BL5R+GqKoyEyEs4XnCekMAVM9uK3eWrNY84CHbjcDjeqxv0Xv3htaQ7fHoyPLO/qkDHFWxVzYsA//fMiireRKoYV4yE3MBtciZwP6Iqqw2Pd29zLz2MGQELX5fkqKy+d5/a4VHTFqy67XuRaLOI6Xzp1Zjm7U7ZA7JGCl381bI1ITFUByIulWYB&p=
User-Agent: ZDM/4.0; Windows Mobile 8.0
X-WP-Client-Config-Version: 107
X-WP-MO-Config-Version: 803
X-WP-Device-ID: A7DF71A4B57D13FB1B904CAA83FE02B63B8EF2457E02A2166C0686D22C200000
X-WP-ANID: A=264684E3878630980823C9A1FFFF0000&E=ea9&W=1
Host: marketplaceedgeservice.windowsphone.com
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/atom+xml
Vary: Accept-Encoding
X-WP-Activity-ID: f97aa71b-03d9-4d58-ad6d-23ed6d351096
X-WP-Served-By: MarketplaceEdge_IN_2.EUNO.Pod1.9.0.13266.1
X-WP-Latest-Client-Config-Version: 107
Date: Sun, 13 Oct 2013 22:16:22 GMT
Content-Length: 905
<?xml version="1.0" encoding="utf-8"?><a:entry xmlns:a="http://www.w3.org/2005/Atom" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns="http://schemas.zune.net/catalog/music/2007/10"><a:link rel="self" type="application/atom+xml" href="/v8/ratings/product/9b255dee-4682-49d8-88bc-d568e40f84bc/user/reviews" /><a:updated>2013-01-12T15:53:31.097000Z</a:updated><a:author><a:name>Hermit</a:name></a:author><a:id>tag: marketplaceedge.windowsphone.com,2013-10-13:/v8/ratings/product/9b255dee-4682-49d8-88bc-d568e40f84bc/user/reviews</a:id><a:content type="html">Hi this is the developer of this app. Just wanted to apologize for issues over the last few versions. Please email me if you still have issues</a:content><userRating>8</userRating><device>RM-821_eu_euro1_276</device><productVersion>8.0</productVersion><hidden>false</hidden><reviewId>3ba388e8-925a-4326-8329-1e365dc7df5e</reviewId></a:entry>

The findings show that there is a GET method that requires authorisation code and device id / ANID in the http headers.
The response is the user review with the ratings

Update: I was asked to check if the rating / review submitted by user makes it to device. I have tested it with another app and yes the device gets the review after a bit (not immediate.. few mins at least). This means that ANID has a easy way of translating to user identifiable live id :) but we already knew that :)

walking directions with using Uri Navigation #wpdev

In past, I usually opened the map control and set the location to that of the cinema. The user can then tap route option to plot the route.

Recently I decided to use HERE maps launchers to do the same.. however Glenn Versweyveld suggested that I should rather use How to request driving or walking directions for Windows Phone 8.

Using that was equally easy.. you craft a Uri and then you launch it. the system takes care of lauching drive / walk apps appropriately.

Being a good developer, I wanted to assign a name to location shown on maps so I used Destination.Name parameter with something like “Cineworld – London Enfield” being my closest cinema.

When using ms-drive, Nokia HERE Drive+ launched correctly and display destination and driving directions.

However using ms-walk, Nokia HERE Maps would start but show users current location.. nothing about destination.. totally ignored. I tweeted my frustration and Randall Arnold mentioned that he had lots of issues with ms-drive and ms-walk and that it would ignore everything after ‘-’ character.

So I stripped ‘-’ out of destination but still nothing. so I decided to strip blank space ‘ ‘. Instead of specifying the whole “Cineworld – London Enfield”, I just set Destination.Name to “Cineworld”. Bingo.. HERE Maps launches with correct destination and walking directions.

Job done.

WinRT and pointer manipulation #win8dev #winrtdev

We all know how Windows 8 introduced touch as a first class member in user input. If you have used Windows 8 it feels intuitive to touch.. hell I used to touch my non-touch laptop occasionally and not because I wanted to fondle it :)

Rewinding a bit.. in Silverlight and Windows Phone, we had a nice historic Touch.FrameReported event. This was the fastest way to detect pointer / touch interaction on the screen. It also supported multi-touch detection by means of exposing TouchFrameEventArges.GetTouchPoints.

We also know how famously .NET components were sliced and diced (lets leave it for some other time when I have had a drink or two). So its only natural that WinRT 1.0 native XAML didn’t have Touch.FrameReported.

The other high level mechanism for touch event detection was the set of manipulation events
* UIElement.ManipulationStarted
* UIElement.ManipulationDelta and
* UIElement.ManipulationCompleted

These made it to WinRT 1.0 XAML.. However there were changes. Now it comes armed with ManipulationMode.

The MEME RTFM definitely applies to me – I never read the manual… When we had proper MSDN lib for documentation it was all good.. now documentation is all over the place.. (Damn it… need to focus more… another thing to talk over a drink).

So what happened.. why trying to detect touch / pointer events, I started using Manipulation events but hell no.. the event’s just wouldn’t get called.. The mode has tons of options but to handle it yourself you need to set the ManipulationMode to All. Someone remind me something.. If I declare a bleeding event handler.. isn’t my intention already pretty clear ? (Something every logical from .NET lost in translation on its way to WinRT design).

So eventually I got that to work by setting ManipulationMode to All. Surprise surprise, the manipulation event args only expose a few things like Position and PointerType…So this definitely can’t do multi-touch.. sure it will detect all the touch inputs but it can’t tell the difference the two fingers!!

Time to give it the middle one and start looking at alternatives. Previously in WPF / Silverlight we had Mice events
* UIElement.MouseLeftButtonDown
* UIElement.MouseLeftButtonUp
* UIElement.MouseMove

Those didn’t directly made it. WinRT merged Mice and Touch events into Pointer events. So now we have
* UIElement.PointerPressed
* UIElement.PointerReleased
* UIElement.PointerMoved

The PointerRoutedEventArgs that are passed to the event handers not only give you current position and intermediate positions, they also give you access to Pointer associated with the event. Each touch point / mice button gets a unique id. This way you can detect activity of each touch points over the screen.

The post is very verbose and lot of gibberish.. let me throw some code snippet to lighten it up :)

private void LayoutRoot_PointerMoved(object sender, PointerRoutedEventArgs e)
{
    Point position2 = e.GetCurrentPoint(this.LayoutRoot).Position;
    Pointer pointer2 = e.Pointer;
    FreeSketch(position2, pointer2);
}

private void FreeSketch(Point position2, Pointer pointer2)
{
    if (bDraw && pointer1.PointerId == pointer2.PointerId)
    {
        Line l = new Line()
        {
            X1 = position1.X,
            Y1 = position1.Y,
            X2 = position2.X,
            Y2 = position2.Y,
            StrokeThickness = 5,
            Stroke = new SolidColorBrush(Colors.Blue),
            Fill = new SolidColorBrush(Colors.Blue)
        };
        this.LayoutRoot.Children.Add(l); // adding to canvas
        this.position1 = position2;
    }
}

Happy coding

Search charm and user input gotchas #win8dev #winrtdev

A while back I implemented external and in-app search using search charm. This means that from anywhere in the app or even from main metro start screen, the user can start typing (which causes search to popup) and assuming they choose Cineworld, the search is passed off to my app.

Fast forward about 2 months back when I was implementing user submitted reviews for films and cinemas. I realised that the text input wouldn’t accept a single character – instead would keep showing the search charm….

aayyee.. that’s the search implementation… so I needed to selectively disable it depending upon where user is and what the user is up to.. trying to implement something meant going to layoutaware page so I don’t have to keep doing this.. whilst I was trying to figure out what to do next, I came across AllowSearch method.. strange.. then it occurred to me that I had added that while I was implementing search

 

protected void AllowSearch(bool bAllowed)
{
    SearchPane.GetForCurrentView().ShowOnKeyboardInput = bAllowed;
}

 
so I went about calling this.AllowSearch(false); in pages where I didn’t want search enabled.

Fast forward a bit closer, last week, I was using WebView allow ticket purchase in-app.. I tried clicks etc but not the full login etc as I know that already works and that’s implemented by Cineworld themselves.. I submit that app and a user contacts me saying.. they can’t book tickets.. search charm keeps popping up…

Damn it.. I had totally forgotten about AllowSearh method. Lets hope this serves as a reminder to myself to remember this in future.
Happy coding