silverlight

Funky Popups in Silverlight 3

An old post of mine, Funky Popups using Tweener, showed how you could use the Tweener library in SilverlightContrib to make some interesting transitions.

When Silverlight 3 was released, they added these kinds of tweening functions into the core runtime, so it’s probably time to look at how to use them.

In the original code, our storyboard for one of the items looked like this:


<DoubleAnimationUsingKeyFrames
 Storyboard.TargetName="bounceScaleTransform"
 Storyboard.TargetProperty="ScaleX"
 tween:Tween.From="0.6"
 tween:Tween.To="1"
 tween:Tween.Fps="10"
 tween:Tween.TransitionType="EaseOutBounce"
 Duration="0:0:0.5"/>

Here’s how the equivalent using Silverlight 3’s easing functions. One important different is that we’re not forced to use a keyframe animation any more.


<DoubleAnimation
 Storyboard.TargetName="bounce3ScaleTransform"
 Storyboard.TargetProperty="ScaleX"
 From="0.6"
 To="1"
 Duration="0:0:0.5">
 <DoubleAnimation.EasingFunction>
 <BounceEase EasingMode="EaseOut" Bounces="1" Bounciness="4"/>
 </DoubleAnimation.EasingFunction>
 </DoubleAnimation>

In place of the tweener attributes is the EasingFunction which can be one of several kinds. To simulate the old bounce behaviour we’ve used a BounceEase function and set EaseOut for the easing mode. The attributes Bounces and Bounciness can adjust the amount of bounce. These values get close to the Tweener behaviour and are fine for our behaviour.

The Swish animation can be similarly replaced by BounceEase animations.

The updated project can be downloaded from here.

Advertisements

Why don’t I get MouseUp events in the Bing Maps Silverlight Component

The Bing Maps Silverlight component (currently in CTP) allows you to overlay items on the map, but it does some odd things to mouse events. Take the following Xaml:


<UserControl x:Class="BingBlogPost.MainPage"
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
 xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
 xmlns:map="clr-namespace:Microsoft.VirtualEarth.MapControl;assembly=Microsoft.VirtualEarth.MapControl"
 mc:Ignorable="d" d:DesignWidth="640" d:DesignHeight="480">
 <Grid x:Name="LayoutRoot">
 <map:Map>
 <Border Padding="8"
 Background="Gray"
 HorizontalAlignment="Center"
 VerticalAlignment="Center"
 MouseLeftButtonUp="Message_Click"
 >
 <TextBlock Text="Click here"/>
 </Border>
 </map:Map>
 </Grid>
</UserControl>

This will show the world map, with our ‘control’ in the centre. Now, mouse handling is interesting using the map component. You’ll notice that if you click and drag on the control, the map underneath will drag. In this example we’ve got an event handler which is set to receive the button up event (to simulate a mouse click). here’s the event handling code:

private void Message_Click(object sender, MouseButtonEventArgs e)
{
     MessageBox.Show(“Clicked!”);
}

But you’ll notice that this event doesn’t fire on a single click. Oddly, it does fire after a double-click. So clearly, the map itself is doing something tricky with events.

If you want to get a click event, there’s two things you can do. You could put in an actual button. This handles mouse events properly and will pass on its own Click event as you’d expect. But if you want to roll your own, there’s a simple trick: add a handler on the control for MouseLeftButtonDown and handle it with this code:

private void Message_MouseDown(object sender, MouseButtonEventArgs e)
{
     e.Handled = true;
}

This is enough to stop the map eating the mouse down event and capturing the mouse (which is why you never get the event). You’ll get the Up event as you expect and you can handle it normally.

Funky Popups using Tweener

Please note: There is an update to this post, Funky Popups using Silverlight 3, which describes how to achieve the same effect using the built-in easing functions of silverlight 3. However, this post remains a good desription of what easing does.

On a current project, we have some controls which have to pop up more info when clicked or using MouseOver. Because this is Silverlight, obviously we want it to look nice, so just switching Visibility from Collapsed to Visible doesn’t really cut the mustard. And I feel there are definite benefits from having some form of animation which conceptually links the box that pops up with the object you used to pop it up – this has been the case right back to the first Mac when new windows would grow and shrink accompanied by a simple animated border. I’ve been playing with various ways to animate these transitions, including using the Tweener class from the SilverlightContrib project.

In Silverlight we’ve got a lot of ways we can make information appear – sliding, fading in, scaling, and my default ‘appearance’ animation tended to be a simple fade combined with a linear scale.

Here’s a XAML for an illustration of this.


<UserControl x:Class="FunkyPopup.Page"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
>
<Grid x:Name="LayoutRoot" Background="White">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
<ColumnDefinition Width="*"/>
</Grid.ColumnDefinitions>
<Grid Grid.Column="0"
MouseEnter="simple_MouseEnter"
MouseLeave="simple_MouseLeave">
<Grid.Resources>
<Storyboard x:Name="simpleFadeBoardIn">
<DoubleAnimation
Storyboard.TargetName="simpleScaleTransform"
Storyboard.TargetProperty="ScaleX"
From="0.6"
To="1"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="simpleScaleTransform"
Storyboard.TargetProperty="ScaleY"
From="0.6"
To="1"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="simpleBorder"
Storyboard.TargetProperty="Opacity"
From="0"
To="1"
Duration="0:0:0.5"/>
</Storyboard>
<Storyboard x:Name="simpleFadeBoardOut">
<DoubleAnimation
Storyboard.TargetName="simpleScaleTransform"
Storyboard.TargetProperty="ScaleX"
From="1"
To="0.6"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="simpleScaleTransform"
Storyboard.TargetProperty="ScaleY"
From="1"
To="0.6"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="simpleBorder"
Storyboard.TargetProperty="Opacity"
From="1"
To="0"
Duration="0:0:0.5"/>
</Storyboard>
</Grid.Resources>
<Border VerticalAlignment="Center" HorizontalAlignment="Center" BorderThickness="3" BorderBrush="Black" Background="White" CornerRadius="5">
<TextBlock Margin="8" Text="Simple Fade and Scale"/>
</Border>
<Border
x:Name="simpleBorder"
VerticalAlignment="Center"
HorizontalAlignment="Center"
BorderThickness="3"
BorderBrush="Black"
Background="White"
CornerRadius="5"
RenderTransformOrigin="0.5,0.5">
<Border.RenderTransform>
<TransformGroup>
<ScaleTransform x:Name="simpleScaleTransform" ScaleX="0" ScaleY="0"/>
</TransformGroup>
</Border.RenderTransform>
<TextBlock Margin="8" Width="250" TextWrapping="Wrap" Text="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque eu lacus in magna posuere sagittis. Curabitur vel odio nec enim mollis cursus. Etiam turpis. Nullam risus nisl, semper et, facilisis vitae, ultricies sed, quam. Sed pharetra scelerisque est. Vivamus lorem. Sed id elit sed mi ultrices feugiat. Mauris luctus tellus. Nullam vestibulum porta lorem. Sed congue pellentesque ipsum. Praesent dolor. Aliquam leo felis, euismod at, adipiscing at, auctor quis, urna."/>
</Border>
</Grid>
</Grid>
</UserControl>

What this does is put two Borders containing text in a grid column. One is the ‘control’ and the other is the information we want to pop up. The popup border has a name and a ScaleTransform with both scales set to 0. Then there are two storyboards, one to make the info pop up, the other to hide it again.

The storyboards animate the opacity, and also scale up from 0.7 to 1, giving a fairly pleasant effect when it appears. The transition takes half a second, a little longer than you’d actually want in real life, but this is intended to demonstrate the various options in  animating, rather than be a finished item. Slower transitions make it easier to see what’s happening.

The associated code-behind is here, and merely handles the MouseEnter and MouseLeave events, firing the appropriate storyboards.


using System.Windows.Controls;
using System.Windows.Input;

namespace FunkyPopup
{
public partial class Page : UserControl
{
public Page()
{
InitializeComponent();
}

private void simple_MouseEnter(object sender, MouseEventArgs e)
{
simpleFadeBoardIn.Begin();
}

private void simple_MouseLeave(object sender, MouseEventArgs e)
{
simpleFadeBoardOut.Begin();
}
}
}

If you run this code, you’ll get a border/text ‘control’. If you move the mouse over the control the info pane will pop up, and it will vanish when the mouse leaves the control. You’ll notice I set the RenderTransformOrigin property on the info grid so it scales from the centre, otherwise it scales from the top left of the grid.

So far, this is just a simple linear animation. Looks fine but there are some simple ways to do a bit more.

Tweening

A lot of animation makes use of tweening which is a process of taking the startpoint and endpoint of a desired animation and generating the steps in between. The whole Silverlight animation process does this for you whenever you use

DoubleAnimation

for example. It takes the From and To values, sees how long the animation should take, and each time a frame needs to be drawn, it mathematically calculates what the ‘in-between’ value should be – halfway through the animation the value will be the mean of the first and last value.

That’s fine for quite a lot of animation, but often you want something more – perhaps you want the animation to start quickly and slow down as it nears the end. Or vice versa. Silverlight has a built-in way to achieve some if these effects – see the SplineDoubleKeyFrame class for a pointer to how it works – but I’m looking at a different way of achieving these results.

What we want is a way of saying ‘Here’s the start point, and the end point, please animate between these points using a specific type of ‘in-between’ calculation. This is what the Tweener class does for you. I’ll show an example that would be difficult to do otherwise, then discuss what it’s doing.

To use the Tweener library, you’ll have to first download the SilverlightContrib dlls (get the source as well if you’re interested) and add a reference to the SilverlightContrib.dll in your project. Then in your XAML you’ll need to add a new namespace like so:


xmlns:tween="clr-namespace:SilverlightContrib.Tweener;assembly=SilverlightContrib"

I add a new ‘control’ to the test application, in the next grid column like so:


<Grid Grid.Column="1"
MouseEnter="bounce_MouseEnter"
MouseLeave="bounce_MouseLeave">
<Grid.Resources>
<Storyboard x:Name="bounceFadeBoardIn">
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="bounceScaleTransform"
Storyboard.TargetProperty="ScaleX"
tween:Tween.From="0.6"
tween:Tween.To="1"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.5"/>
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="bounceScaleTransform"
Storyboard.TargetProperty="ScaleY"
tween:Tween.From="0.6"
tween:Tween.To="1"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="bounceBorder"
Storyboard.TargetProperty="Opacity"
From="0"
To="1"
Duration="0:0:0.5"/>
</Storyboard>
<Storyboard x:Name="bounceFadeBoardOut">
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="bounceScaleTransform"
Storyboard.TargetProperty="ScaleX"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseInBounce"
tween:Tween.From="1"
tween:Tween.To="0.6"
Duration="0:0:0.5"/>
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="bounceScaleTransform"
Storyboard.TargetProperty="ScaleY"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseInBounce"
tween:Tween.From="1"
tween:Tween.To="0.6"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="bounceBorder"
Storyboard.TargetProperty="Opacity"
From="1"
To="0"
Duration="0:0:0.5"/>
</Storyboard>
</Grid.Resources>
<Border VerticalAlignment="Center" HorizontalAlignment="Center" BorderThickness="3" BorderBrush="Black" Background="White" CornerRadius="5">
<TextBlock Margin="8" Text="Bounce Scale and fade"/>
</Border>
<Border
x:Name="bounceBorder"
VerticalAlignment="Center"
HorizontalAlignment="Center"
BorderThickness="3"
BorderBrush="Black"
Background="White"
CornerRadius="5"
RenderTransformOrigin="0.5,0.5">
<Border.RenderTransform>
<TransformGroup>
<ScaleTransform x:Name="bounceScaleTransform" ScaleX="0" ScaleY="0"/>
</TransformGroup>
</Border.RenderTransform>
<TextBlock Margin="8" Width="250" TextWrapping="Wrap" Text="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque eu lacus in magna posuere sagittis. Curabitur vel odio nec enim mollis cursus. Etiam turpis. Nullam risus nisl, semper et, facilisis vitae, ultricies sed, quam. Sed pharetra scelerisque est. Vivamus lorem. Sed id elit sed mi ultrices feugiat. Mauris luctus tellus. Nullam vestibulum porta lorem. Sed congue pellentesque ipsum. Praesent dolor. Aliquam leo felis, euismod at, adipiscing at, auctor quis, urna."/>
</Border>
</Grid>

and the event handlers in the C# file:


private void bounce_MouseEnter(object sender, MouseEventArgs e)
{
bounceFadeBoardIn.Begin();
}

private void bounce_MouseLeave(object sender, MouseEventArgs e)
{
bounceFadeBoardOut.Begin();
}

If you run it, you’ll see that the info pops up quicker and gives a little bounce before it settles down into its 100% size and position. So what’s it doing? And how?

The key difference between the two storyboards is that the second ones are DoubleAnimationUsingKeyframes instead of the simpler DoubleAnimation. But you’ll also notice that I’m not supplying any keyframes as you would normally expect. That’s what Tweener does for you. I replaced the From and To properties with:


tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseInBounce"
tween:Tween.From="1"
tween:Tween.To="0.6"

These are Attached Properties supplied by the Tween class, and what happens when these properties are attached to the DoubleAnimationUsingKeyframes timeline is that the Tweener library will generate the required keyframes for the required animation for you. It needs to know the From and To values, and you have to specify the number of frames per second to generate but this merely tells the tweener how many keyframes to generate – it doesn’t affect the ultimate smoothness of the animation since each generated keyframe will also have extra in-between frames generated for it by the Silverlight animation system. The really interesting one is the TransitionType. This specifies which equation to use out of a selection of possible ways to transition from one value to another. In this example we’re using the EaseOutBounce type for the appearance and EaseInBounce for the vanishing.

The list of types is fairly long (intellisense lists them) and their naming tells you something about them. EaseInXXX means the particular type of effect happens at the start of the animation. EaseOutXXX means it happens at the end (which is why we use EaseOutBounce for the popup appearing). You could try changing the transition type in the XAML to see what effect each of them gives. Generally the transitions are designed to ease the animation. A simple, linear animation looks very artificial. Items start and stop very suddenly, and there’s no sense of acceleration or deceleration. It’s this effect which easing is designed to solve. Most of the easing functions go straight from the first value to the last, with varying extremes of acceleration (and by using EaseInOutXXX you can get the acceleration both at the start and the end) and others, like the EaseOutBounce we used here, give a slightly funkier effect where the animation overshoots the endpoint, then bounces back and forth a little. It gives a more organic feel than a linear animation.

Some of these easing effects can be replicated using the KeySpline property of SplineDoubleKeyFrame, but it can be hard to know exactly what shape spline curve to use to achieve the required effect. These transitions originated in the Flash world, where they are quite commonly used, as it’s much easier to select the type of transition by name. Also, a simple spline can’t replicate the bouncy effects.

So, now we’ve got the box appearing with a little more wit, is there anything else we can do? Well, you can get some even nicer effects simply by playing with the timings of the animations, making the scaling asymmetric. I’ll add yet another of our ‘controls’. Here’s the full XAML for the new item:


<Grid Grid.Column="2"
MouseEnter="swish_MouseEnter"
MouseLeave="swish_MouseLeave">
<Grid.Resources>
<Storyboard x:Name="swishFadeBoardIn">
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleX"
tween:Tween.From="0.3"
tween:Tween.To="1"
tween:Tween.Fps="10"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.5"
BeginTime="0:0:0.2"/>
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleY"
tween:Tween.From="0"
tween:Tween.To="1"
tween:Tween.Fps="10"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.5"/>
<DoubleAnimation
Storyboard.TargetName="swishBorder"
Storyboard.TargetProperty="Opacity"
From="0"
To="1"
Duration="0:0:0.1"/>
</Storyboard>
<Storyboard x:Name="swishFadeBoardOut">
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleX"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseInBounce"
tween:Tween.From="1"
tween:Tween.To="0.3"
Duration="0:0:0.3"/>
<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleY"
tween:Tween.Fps="30"
tween:Tween.TransitionType="EaseInBounce"
tween:Tween.From="1"
tween:Tween.To="0"
Duration="0:0:0.3"
BeginTime="0:0:0.2"/>
<DoubleAnimation
Storyboard.TargetName="swishBorder"
Storyboard.TargetProperty="Opacity"
From="1"
To="0"
BeginTime="0:0:0.45"
Duration="0:0:0.05"/>
</Storyboard>
</Grid.Resources>
<Border VerticalAlignment="Center" HorizontalAlignment="Center" BorderThickness="3" BorderBrush="Black" Background="White" CornerRadius="5">
<TextBlock Margin="8" Text="Swish Scale and fade"/>
</Border>
<Border
x:Name="swishBorder"
VerticalAlignment="Center"
HorizontalAlignment="Center"
BorderThickness="3"
BorderBrush="Black"
Background="White"
CornerRadius="5"
RenderTransformOrigin="0.5,0.5">
<Border.RenderTransform>
<TransformGroup>
<ScaleTransform x:Name="swishScaleTransform" ScaleX="0.3" ScaleY="0"/>
</TransformGroup>
</Border.RenderTransform>
<TextBlock Margin="8" Width="250" TextWrapping="Wrap" Text="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque eu lacus in magna posuere sagittis. Curabitur vel odio nec enim mollis cursus. Etiam turpis. Nullam risus nisl, semper et, facilisis vitae, ultricies sed, quam. Sed pharetra scelerisque est. Vivamus lorem. Sed id elit sed mi ultrices feugiat. Mauris luctus tellus. Nullam vestibulum porta lorem. Sed congue pellentesque ipsum. Praesent dolor. Aliquam leo felis, euismod at, adipiscing at, auctor quis, urna."/>
</Border>
</Grid>

And the inevitable event handlers in the csharp file:


private void swish_MouseEnter(object sender, MouseEventArgs e)
{
swishFadeBoardIn.Begin();
}

private void swish_MouseLeave(object sender, MouseEventArgs e)
{
swishFadeBoardOut.Begin();
}

This time I’ll delve a bit deeper into the slight changes I’ve made get the effect. What I wanted was for the info pane to appear as if it’s first being stretched vertically, then horizontally. So I needed to change the timing and duration of the scaling animations. Here’s the animation for the X scale factor when it’s appearing.


<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleX"
tween:Tween.From="0.3"
tween:Tween.To="1"
tween:Tween.Fps="10"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.3"
BeginTime="0:0:0.2"/>

The key changes here are:

  • The From value starts at 0.3 instead of 0. If we left it at 0, the first part of the Y animation would be invisible because it would be stretching a zero-width block.
  • The BeginTime and Duration have been adjusted so that it doesn’t start until 0.2 secs into the animation, so the first part of the animation is a pure vertical stretch

<DoubleAnimationUsingKeyFrames
Storyboard.TargetName="swishScaleTransform"
Storyboard.TargetProperty="ScaleY"
tween:Tween.From="0"
tween:Tween.To="1"
tween:Tween.Fps="10"
tween:Tween.TransitionType="EaseOutBounce"
Duration="0:0:0.5"/>

The Y stretch runs for the full half second, but it starts at 0 so the box does appear from nowhere.


<DoubleAnimation
Storyboard.TargetName="swishBorder"
Storyboard.TargetProperty="Opacity"
From="0"
To="1"
Duration="0:0:0.1"/>

The Opacity animation has been made much faster, so it completes almost immediately, so we get to see the full effect of the stretching. Without this, part of the initial appearance would not really be visible.

The inverse animation is fairly straightforward, it’s just a matter of inverting the timings, so we start the X scale immediately, but the duration remains 0.3s. The Y scale keeps its full duration (although you can vary the effect by playing with the length and start point of the Y scale as well). Because we’re scaling down, we use the EaseInBounce instead of the EaseOutBounce we used for the appearance.

One current disadvantage of using Tweener is that Blend doesn’t recognise it, so there’s no tool support for it. I have some ideas of ways around that, but this post is already far too long, so I’ll save those for another day.

Here’s the full source code for this project.

Why does LayoutUpdated pass a null sender?

Because it does not mean what you think it means.

As Dave Relyea explains, LayoutUpdated is fired every time layout did anything in the tree so it’s not specific to a particular object, and you can’t use the sender to tell you what you’re interested in.

What you need is the SizeChanged event. LayoutUpdated is going to get fired an awful lot, while SizeChanged gets fired when the object you’re interested in has its size changed. It won’t get fired if the position of the object changes, but that’s probably what you want most of the time.

My particular code had a UserControl in a popup, and the UserControl had a variable size. I wanted it to always remain on screen, intelligently positioning itself so that it’s fully on screen (like a menu would) but when it’s first created (and indeed, laid out) its actual size is {0,0} because (I believe) that it hasn’t been added to the visual tree yet because the parent Popup is closed. I needed to wait until the size of the object was fully calculated before I positioned the popup, and I thought LayoutUpdated would do the trick, so I had some code like this:

void infoPane_LayoutUpdated(object sender, EventArgs e)
{
WalkInfoPane pane = sender as WalkInfoPane;
if (pane != null && pane.ActualHeight > 0 && pane.ActualWidth > 0)
{
Point anchor = PopupAnchor.TransformToVisual(null).Transform(new Point());
if (anchor.Y < 240) { PanePopup.VerticalOffset = -240 + (240 - anchor.Y); } else { PanePopup.VerticalOffset = -240; } pane.MakeVisible(); } } [/sourcecode] (PanePopup is the parent Popup. I was expecting the sender to point at the child control, but it gives null.) Not surprisingly, this code was never getting called because sender is always null. Using the same code in a SizeChanged event works perfectly. It's a shame MSDN doesn't cover details like this.

ScottGu on Silverlight 3

Scott Guthrie, head of virtually everything developy at Microsoft, has just posted a blog on Silverlight, with some new information about Silverlight 3. For me, the most interesting items are:

  • Silverlight now on over 1 in 4 browsers in some form (1.0 and 2, I presume). This is the first concrete number we’ve had about Silverlight penetration. To be honest, I’ve no idea what constitutes a good number, since presumably anything less than Flash’s penetration would be seen as a negative, but at this stage it’s not a bad figure.
  • 3D support and GPU acceleration is coming in SL3 (along with H.264, which had already been announced). The GPU use is very good news for performance reasons. 3D support is a nice-to-have for me, but not killer. But it will make some UI features easier to do.
  • Richer data-binding and more controls. More controls are always nice, and let’s hope that some of WPF’s databinding abilities finally make it to Silverlight. Being able to bind controls to controls directly, for example, would make some things a lot easier.

Of course, we’ve got to wait for these new goodies, but it’s nice to see them coming. Hopefully a CTP or Beta will be around before too long.

A simple Library to Build a Deep Zoom Image

My current project requires a lot of work with Deep Zoom images. We recently received some very high-res photos, around 500M tiff files, some of which Deep Zoom Composer was unable to process. My first thought was to split them into smaller pieces and compose them together, but this led to visible join lines when processed, possibly due to rounding issues.

I’d already written some code to generate parts of a much larger deep zoom image, which was focused on building or rebuilding a small portion of much larger composition (over 2.5 million tiles in total) but it wasn’t quite ready to build a whole DZI.

So I took the bits of the code I needed and put them in a class by themselves. It’s very simple, and only designed to take a single source image and create a deep zoom tileset which can then be used by Silverlight.

It’s fairly unsophisticated in how it works. It loads in the source image (so you need enough RAM to hold that image at least) then renders out the top level tiles. Then it dumps the source image and renders the next levels down, rendering each one using the tiles of the level above. This seems to give a good result and was a technique I used previously when I was stitching together many source images and finding that DZC would leave ugly join lines at certain zoom levels.

The code feels a bit hacky, since most of the calculations were rough guesses or trial and error, but it definitely works for a single image, and it might be useful to someone, maybe as a server-side component to build deep zoom images on the fly. It’s biggest drawback is that it takes a long time, but if you’re using it in a WPF app you can always run it using a BackgroundWorker.

Note that this is WPF code, not Silverlight.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Windows.Media.Imaging;
using Path = System.IO.Path;
using System.Windows;
using System.Windows.Media;
using System.Xml.Linq;

namespace DeepZoomBuilder
{
	public enum ImageType
	{
		Png,
		Jpeg
	}

	public class DeepZoomCreator
	{
		/// <summary>
		/// Default public constructor
		/// </summary>
		public DeepZoomCreator() { }

		/// <summary>
		/// Create a deep zoom image from a single source image
		/// </summary>
		///
<param name="sourceImage">Source image path</param>
		///
<param name="destinationImage">Destination path (must be .dzi or .xml)</param>
		public void CreateSingleComposition(string sourceImage, string destinationImage, ImageType type)
		{
			imageType = type;
			string source = sourceImage;
			string destDirectory = Path.GetDirectoryName(destinationImage);
			string leafname = Path.GetFileNameWithoutExtension(destinationImage);
			string root = Path.Combine(destDirectory, leafname); ;
			string filesdir = root + "_files";

			Directory.CreateDirectory(filesdir);
			BitmapImage img = new BitmapImage(new Uri(source));
			double dWidth = img.PixelWidth;
			double dHeight = img.PixelHeight;
			double AspectRatio = dWidth / dHeight;

			// The Maximum level for the pyramid of images is
			// Log2(maxdimension)

			double maxdimension = Math.Max(dWidth, dHeight);
			double logvalue = Math.Log(maxdimension, 2);
			int MaxLevel = (int)Math.Ceiling(logvalue);
			string topleveldir = Path.Combine(filesdir, MaxLevel.ToString());

			// Create the directory for the top level tiles
			Directory.CreateDirectory(topleveldir);

			// Calculate how many tiles across and down
			int maxcols = img.PixelWidth / 256;
			int maxrows = img.PixelHeight / 256;

			// Get the bounding rectangle of the source image, for clipping
			Rect MainRect = new Rect(0, 0, img.PixelWidth, img.PixelHeight);
			for (int j = 0; j <= maxrows; j++)
			{
				for (int i = 0; i <= maxcols; i++)
				{
					// Calculate the bounds of the tile
					// including a 1 pixel overlap each side
					Rect smallrect = new Rect((double)(i * 256) - 1, (double)(j * 256) - 1, 258.0, 258.0);

					// Adjust for the rectangles at the edges by intersecting
					smallrect.Intersect(MainRect);

					// We want a RenderTargetBitmap to render this tile into
					// Create one with the dimensions of this tile
					RenderTargetBitmap outbmp = new RenderTargetBitmap((int)smallrect.Width, (int)smallrect.Height, 96, 96, PixelFormats.Pbgra32);
					DrawingVisual visual = new DrawingVisual();
					DrawingContext context = visual.RenderOpen();

					// Set the offset of the source image into the destination bitmap
					// and render it
					Rect rect = new Rect(-smallrect.Left, -smallrect.Top, img.PixelWidth, img.PixelHeight);
					context.DrawImage(img, rect);
					context.Close();
					outbmp.Render(visual);

					// Save the bitmap tile
					string destination = Path.Combine(topleveldir, string.Format("{0}_{1}", i, j));
					EncodeBitmap(outbmp, destination);

					// null out everything we've used so the Garbage Collector
					// knows they're free. This could easily be voodoo since they'll go
					// out of scope, but it can't hurt.
					outbmp = null;
					context = null;
					visual = null;
				}
				GC.Collect();
				GC.WaitForPendingFinalizers();
			}

			// clear the source image since we don't need it anymore
			img = null;
			GC.Collect();
			GC.WaitForPendingFinalizers();

			// Now render the lower levels by rendering the tiles from the level
			// above to the next level down
			for (int level = MaxLevel - 1; level >= 0; level--)
			{
				RenderSubtiles(filesdir, dWidth, dHeight, MaxLevel, level);
			}

			// Now generate the .dzi file

			string format = "png";
			if (imageType == ImageType.Jpeg)
			{
				format = "jpg";
			}

			XElement dzi = new XElement("Image",
				new XAttribute("TileSize", 256),
				new XAttribute("Overlap", 1),
				new XAttribute("Format", format), // xmlns="http://schemas.microsoft.com/deepzoom/2008">
				new XElement("Size",
					new XAttribute("Width", dWidth),
					new XAttribute("Height", dHeight)),
				new XElement("DisplayRects",
					new XElement("DisplayRect",
						new XAttribute("MinLevel", 1),
						new XAttribute("MaxLevel", MaxLevel),
						new XElement("Rect",
							new XAttribute("X", 0),
							new XAttribute("Y", 0),
							new XAttribute("Width", dWidth),
							new XAttribute("Height", dHeight)))));
			dzi.Save(destinationImage);

		}

		/// <summary>
		/// Save the output bitmap as either Png or Jpeg
		/// </summary>
		///
<param name="outbmp">Bitmap to save</param>
		///
<param name="destination">Path to save to, without the file extension</param>
		private void EncodeBitmap(RenderTargetBitmap outbmp, string destination)
		{
			if (imageType == ImageType.Png)
			{
				PngBitmapEncoder encoder = new PngBitmapEncoder();
				encoder.Frames.Add(BitmapFrame.Create(outbmp));
				FileStream fs = new FileStream(destination + ".png", FileMode.Create);
				encoder.Save(fs);
				fs.Close();
			}
			else
			{
				JpegBitmapEncoder encoder = new JpegBitmapEncoder();
				encoder.QualityLevel = 95;
				encoder.Frames.Add(BitmapFrame.Create(outbmp));
				FileStream fs = new FileStream(destination + ".jpg", FileMode.Create);
				encoder.Save(fs);
				fs.Close();
			}
		}

		/// <summary>
		/// Specifies the output filetype
		/// </summary>
		ImageType imageType = ImageType.Jpeg;

		/// <summary>
		/// Render the subtiles given a fully rendered top-level
		/// </summary>
		///
<param name="subfiles">Path to the xxx_files directory</param>
		///
<param name="imageWidth">Width of the source image</param>
		///
<param name="imageHeight">Height of the source image</param>
		///
<param name="maxlevel">Top level of the tileset</param>
		///
<param name="desiredlevel">Level we want to render. Note it requires
		/// that the level above this has already been rendered.</param>
		private void RenderSubtiles(string subfiles, double imageWidth, double imageHeight, int maxlevel, int desiredlevel)
		{
			string formatextension = ".png";
			if (imageType == ImageType.Jpeg)
			{
				formatextension = ".jpg";
			}
			int uponelevel = desiredlevel + 1;
			double desiredfactor = Math.Pow(2, maxlevel - desiredlevel);
			double higherfactor = Math.Pow(2, maxlevel - (desiredlevel + 1));
			string renderlevel = Path.Combine(subfiles, desiredlevel.ToString());
			Directory.CreateDirectory(renderlevel);
			string upperlevel = Path.Combine(subfiles, (desiredlevel + 1).ToString());

			// Calculate the tiles we want to translate down
			Rect MainBounds = new Rect(0, 0, imageWidth, imageHeight);
			Rect OriginalRect = new Rect(0, 0, imageWidth, imageHeight);

			// Scale down this rectangle to the scale factor of the level we want
			MainBounds.X = Math.Ceiling(MainBounds.X / desiredfactor);
			MainBounds.Y = Math.Ceiling(MainBounds.Y / desiredfactor);
			MainBounds.Width = Math.Ceiling(MainBounds.Width / desiredfactor);
			MainBounds.Height = Math.Ceiling(MainBounds.Height / desiredfactor);

			int lowx = (int)Math.Floor(MainBounds.X / 256);
			int lowy = (int)Math.Floor(MainBounds.Y / 256);
			int highx = (int)Math.Floor(MainBounds.Right / 256);
			int highy = (int)Math.Floor(MainBounds.Bottom / 256);

			for (int x = lowx; x <= highx; x++)
			{
				for (int y = lowy; y <= highy; y++)
				{
					Rect smallrect = new Rect((double)(x * 256) - 1, (double)(y * 256) - 1, 258.0, 258.0);
					smallrect.Intersect(MainBounds);
					RenderTargetBitmap outbmp = new RenderTargetBitmap((int)smallrect.Width, (int)smallrect.Height, 96, 96, PixelFormats.Pbgra32);
					DrawingVisual visual = new DrawingVisual();
					DrawingContext context = visual.RenderOpen();

					// Calculate the bounds of this tile

					Rect rect = smallrect;
					// This is the rect of this tile. Now render any appropriate tiles onto it
					// The upper level tiles are twice as big, so they have to be shrunk down

					Rect scaledRect = new Rect(rect.X * 2, rect.Y * 2, rect.Width * 2, rect.Height * 2);
					for (int tx = lowx * 2; tx <= highx * 2 + 1; tx++)
					{
						for (int ty = lowy * 2; ty <= highy * 2 + 1; ty++)
						{
							// See if this tile overlaps
							Rect subrect = GetTileRectangle(tx, ty);
							if (scaledRect.IntersectsWith(subrect))
							{
								subrect.X -= scaledRect.X;
								subrect.Y -= scaledRect.Y;
								RenderTile(context, Path.Combine(upperlevel, tx.ToString() + "_" + ty.ToString() + formatextension), subrect);
							}
						}
					}
					context.Close();
					outbmp.Render(visual);

					// Render the completed tile and clear all resources used
					string destination = Path.Combine(renderlevel, string.Format(@"{0}_{1}", x, y));
					EncodeBitmap(outbmp, destination);
					outbmp = null;
					visual = null;
					context = null;
				}
				GC.Collect();
				GC.WaitForPendingFinalizers();
			}

		}

		/// <summary>
		/// Get the bounds of the given tile rectangle
		/// </summary>
		///
<param name="x">x index of the tile</param>
		///
<param name="y">y index of the tile</param>
		/// <returns>Bounding rectangle for the tile at the given indices</returns>
		private static Rect GetTileRectangle(int x, int y)
		{
			Rect rect = new Rect(256 * x - 1, 256 * y - 1, 258, 258);
			if (x == 0)
			{
				rect.X = 0;
				rect.Width = rect.Width - 1;
			}
			if (y == 0)
			{
				rect.Y = 0;
				rect.Width = rect.Width - 1;
			}

			return rect;
		}

		/// <summary>
		/// Render the given tile rectangle, shrunk down by half to fit the next
		/// lower level
		/// </summary>
		///
<param name="context">DrawingContext for the DrawingVisual to render into</param>
		///
<param name="path">path to the tile we're rendering</param>
		///
<param name="rect">Rectangle to render this tile.</param>
		private void RenderTile(DrawingContext context, string path, Rect rect)
		{
			if (File.Exists(path))
			{
				BitmapImage img = new BitmapImage(new Uri(path));
				rect = new Rect(rect.X / 2.0, rect.Y / 2.0, ((double)img.PixelWidth) / 2.0, ((double)img.PixelHeight) / 2.0);
				context.DrawImage(img, rect);
			}
		}

	}
}

If you do use this for anything, please let me know as I’d love to see it used elsewhere.

Update: This code is now a little redundant, now that the Deep Zoom Composer team have released a DLL to do the same thing.

Using PathGeometry in Silverlight: An Oddity

I hesitate to characterise this as a bug, because other people have reported that it works for them. But here it is in case knowing it happens, and knowing there’s a workaround might be useful.

I’ve got some code which adds segments to a Path’s PathGeometry in response to mouse clicks. My original project actually added them during an animation, but that doesn’t affect the end result. The problem is that new segments (LineSegment or BezierSegment had the same result) didn’t appear on screen until the surface was forced to re-render. Here’s the relevant code:

Path path = null;

        private void LayoutRoot_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
        {
                Point thisPoint = e.GetPosition(LayoutRoot);
                if (path == null)
                {
                        CreateNewPath(thisPoint);
                        path.LayoutUpdated += new EventHandler(path_LayoutUpdated);
                }
                else
                {
                        path.AddLineElement(thisPoint);
                }
        }

        private void CreateNewPath(Point startPoint)
        {
                path = new Path();
                PathGeometry geometry = new PathGeometry();
                path.Data = geometry;
                PathFigureCollection figures = new PathFigureCollection();
                geometry.Figures = figures;
                PathFigure figure = new PathFigure();
                figures.Add(figure);
                figure.StartPoint = startPoint;
                figure.Segments = new PathSegmentCollection();
                path.Stroke = new SolidColorBrush(Colors.Red);
                path.StrokeThickness = 2;
                path.Stretch = Stretch.None;
                LayoutRoot.Children.Add(path);
        }

What I get when this code runs is that the path gets segments added to it, but they only appear when the control surface has to redraw for some other reason. If you insert a kludge like this after you’ve added the segment it should appear immediately:


    path.Opacity = 0.99;

    path.Opacity = 1.0;

Note that you actually have to change the opacity for the kludge to work, presumably because the PropertyChanged callback won’t get triggered otherwise.

Other things that make the new segments appear are resizing the browser window and causing some other redraw event to happen somewhere on the page. I found that if I had a button on the page and I moved the mouse over it, the mouseover animation was enough to make the segments appear. It looks like adding segments to the geometry isn’t spotted as a change which requires the element to be redrawn, and it feels like a bug to me. The identical code in a WPF application works as expected.

Still, the kludge isn’t too much of a hardship, and in general you’re probably going to be changing other visual elements at the same time as you’re adding your path segments, so perhaps it’s not such a big deal, but I wasted some time trying to figure why my code wasn’t behaving as expected, so perhaps this will help others.