Hey! From this Monday to Thursday I’ll be at Unterschleissheim, Germany, attending the Qt Dev Days Munich. Meeting all Qt hackers again will be awesome, as always, but the best thing about this trip will be meeting my INdT’s fellows! ;)
So, if you are aroud:
(*not sure where I took this picture from =/)
Cheers!
There is a lot of effort being put into QtWebKit in order to make it attractive on the mobile front.
Among a tons of bug fixes and good performance improvements there are also lots of new features being developed, mainly geared toward mobile deployment.
The goal with this tutorial is to help you understand some of these new features and how to make the best of them. Or said in other words; how to create a good mobile web view that can be used on touch devices.
First we should make it clear that QGraphicsWebView is the way forward, so if you want to target mobile devices, it is bye bye QWebView. Why is that? Well, the QWebView is based on the QWidget system, thus it cannot easily support rotation, overlays, hardware accelerated compositing and tiling. If you need a QWidget anyway, you can always construct a QGraphicsView (which is a QWidget) with a QGraphicsWebView inside.
So let’s start with the most simple QGraphicsWebView based “browser” ever:
int main(int argc, char **argv)
{
QApplication app(argc, argv);
const int width = 640;
const int height = 480;
QGraphicsScene scene;
QGraphicsView view(&scene);
view.setFrameShape(QFrame::NoFrame);
view.setVerticalScrollBarPolicy(Qt::ScrollBarAlwaysOff);
view.setHorizontalScrollBarPolicy(Qt::ScrollBarAlwaysOff);
QGraphicsWebView webview;
webview.resize(width, height);
webview.load(QUrl("http://www.wouwlabs.com/blogs/jeez"));
scene.addItem(&webview);
view.resize(width, height);
view.show();
return app.exec();
}
Here we just bootstrap a QGraphicsView application and add a QGraphicsWebView to the scene.
It might seem a bit useless as you can only navigate through one website, but it serves well as a simple example. Notice that I’m disabling the scrollbars on the graphics view because QtWebKit handles scrolling and scrollbars automatically. This is due to scrolling optimizations and due to the fact that web authors can interact with the scrollbars for instance style them differently.
On touch-based mobile devices a feature known as tiling is often used. It is used due to the interaction model (touch) as well as a scrolling “optimization”. With this optimization we will have to deal with scrolling ourselves and we thus will have to say good bye to the scrollbar styling. Not a big thing, as mobile browsers usually do not even show scrollbars, but use scroll indicators instead.
Tiling basically means that the contents of the viewport is separated into a grid of tiles, so that when you update some area, instead of just updating the area you actually update the whole tile. This gives a few advantages for scrolling as when you scroll you do not need to repaint the new visible area for each scroll step, as you update a row of tiles each time; tiles that are often only partly on the screen. This minimized all the paint calls that we have to do and makes it possible to make nicely kinetic scrolling a possibility.
Loading, layouting etc are blocking operations. Though barely noticeable on a Desktop machines, these operations can block for a long time on a mobile device, letting the user believe the application has become unresponsive and died. Scrolling which is done by using fingers will also stall and give a bad user experience.
One way to over come this issue, is to do all loading, layouting and painting (basically all non-UI related work) in another thread or process, and just blit the result from the web process/thread to the UI. When using tiles, you can blit any tile available when scrolling. When no tile is available you can show a checkerboard tile instead, not letting the scrolling wait for the tiles to be updated. This results in a responsive interface, with the only disadvantage that you from time to time might see checkerboard tiles.
Tiles also helps with zooming. Repainting at each zoom level change during a zoom animation is basically impossible on a mobile device (or desktop for that sake) and thus with tiling, you can stop the tiles from being updates and just scale the already existing tiles, and then at the end of the animation update tiles on top of the scaled ones.
For now we will ignore the blocking issue and concentrate on the tiling and the interaction model.
Resize to contents
When using tiling, we basically want the QGraphicsWebView to act as our contents, as it supports tiling a.o. things. In order for this we need to make it resize itself to the size of its contents. For this we will use QGraphicsWebView::resizesToContents.
From Qt 4.7 documentation: “If this property is set, the QGraphicsWebView will automatically change its size to match the size of the main frame contents. As a result the top level frame will never have scrollbars. It will also make CSS fixed positioning to behave like absolute positioning with elements positioned relative to the document instead of the viewport.”
This setting, thus, removes the scrollbars for us on the main frame and makes our QGraphicsWebView resize itself to the size of its content.
Enabling it, is as easy as: webview.setResizesToContents(true);
Qt 4.7 docs also says: “This property should be used in conjunction with the QWebPage::preferredContentsSize property. If not explicitly set, the preferredContentsSize is automatically set to a reasonable value.”
If we are going to expand our mobile web view to the size of the contents of its contained page, then that is going to make the view a lot bigger than what can fit on the device’s screen!
Using a view as the window to the contents
The idea is to have a custom widget which has a QGraphicsWebView as a class member. Remember that the QGraphicsWebView will be as big as its content’s size, so this custom widget will serve as a window, as a viewport.
There is not much more to say here, and the following code snippet illustrates it well:
class MobileWebView : public QGraphicsWidget
{
Q_OBJECT
public:
MobileWebView(QGraphicsItem *parent = 0);
~MobileWebView();
bool mousePress(const QPoint &value);
void mouseMove(const QPoint &value);
void mouseRelease(const QPoint &value);
private:
QGraphicsWebView *webView;
};
In order to properly handle mouse events you must install an event filter on web view or stack it behind its parent object (search for QGraphicsItem::ItemStacksBehindParent). By doing this the mouse events will reach a MobileWebView instance before they reach the member QGraphicsWebView. Keep in mind that you’ll need to add some logic in order to distinguish different mouse events and gestures like a single click, double click, click-and-pan, besides scrolling will have to be implemented manually, as zoom, etc. That is left as an exercise to the reader.
Adjusting how contents is laid out
When testing the above on a device, you will notice that many pages do not layout very nicely. In particular the width is larger than the width of the device!
The way web contents is laid out, is that the first the viewport width is used for fitting the contents. If the contents doesn’t fit due to non-flexible element with a width larger than the viewport width, the min width possible will be used. As most pages are written with a desktop browser in mind, that makes only very few sites fit into the width of a mobile device.
QtWebKit has a way to force a layout to a given width/height. What really matters here is the width. If you layout a size to a given width, it will get that width and images etc might get cut of. The width/height is also used for laying out fixed elements, but when we resize the QGraphicsWebView to the size of the contents, fixed elements will not be relative to the view, which is the behaviour found on most mobile browsers.
From Qt 4.7 documentation: “If this property is set to a valid size, it is used to lay out the page.”.
We saw that this property is automatically set to a reasonable value when using QGraphicsWebView::resizesToContents.
As you can imaging, laying out with a smaller viewport can cause pages to break, and as thus, a default value has been chosen so that it almost breaks no pages while still making the sites fit. This value is 960×800. If the device have a bigger resolution, this value can be changed using: webview.page()->setPreferredContentsSize(QSize(desiredWidth, desiredHeight));
You can play around with this and find your own magic number, but let’s stick to this 960px wide for now.
The ‘viewport’ meta tag
As some sites do not work with 960 or want to have control on how the page is laid out, QtWebKit as well as Android, Firefox Mobile and the iPhone Safari supports a meta tag called viewport.
This one also deserves a whole blog post for itself. For now let’s just say that this is a meta tag that Apple came up with to make a web page capable of “telling” the browser how it wants to be shown.
More info here and here, both from Apple’s docs.
In QtWebKit trunk we already have support for this with a nice API. You must connect the signal from QWebPage::viewportChangeRequested(const QWebPage::ViewportHints& hints) to a slot of your mobile web view and use what is provided by QWebPage::ViewportHints to updated your viewport size, scale range, and so on.
This can be tricky and that’s why I’m not going deeper on it right now. Since I know you are curious about it I’ll leave you with one more exercise! So go ahead and try to understand how the guys from MicroB and Firefox Mobile dealt with this.
Enabling the Tiling
We haven’t actually enabled tiling yet, so lets go ahead and do that. That is very simple as it is basically a setting: QWebSettings::globalSettings()->setAttribute(QWebSettings::TiledBackingStoreEnabled, true);
Voila! Mind that if you are going to add animations to your zoom/scale or want to implement a fancy kinetic scrolling
you might want to take a look at QGraphicsWebView::setTiledBackingStoreFrozen. With this you can avoid updates to your
tiles during an animation, for instance.
Avoiding scrollable sub elements
One big issue with the above is that, iframes and sites using frames can contain scrollable sub elements. That doesn’t work well with the touch interaction model, as you want a finger swipe to scroll the whole page and not end up just scrolling a sub frame. Most mobile browser work around this by enabling something called frame flattening.
Going straight to the point: QWebSettings::globalSettings()->setAttribute(QWebSettings::FrameFlatteningEnable, true);
This will make all frames from a web page expand themselves to the size of their contents, keeping us free of scrollable subareas.
Conclusion
As you can see the inner truth here is that there is no “black-underground-magic” at all!
QtWebKit is getting better and better and you have plenty of APIs to help you out developing your applications, be it a browser or a piece of software with embedded web contents.
If you want to take a closer look at some real world examples of QtWebKit in action I’d recommend the raskbrowser and the yberbrowser. I’m sure you will recognize each of the above steps in their source code!
Keep hacking!
… as the subject of the email I received today said. I was so excited when Mat Dobson called me earlier today that I’m still putting my ideas back in order! :)
It’s official! Sketch your World is now one the Nokia’s PUSH N900 winner.
Don’t remember the project?! Check the previous post and/or the PDF and video submitted:
You can follow up the project development on the Push blog.
And now we must run for finishing it up!
jeez
openBossa has released Qt Mobile Demos, a pack of fancy demos for showing off what Qt 4.6 can do!
Until now, there are four of them: MyBudget, ShoppingList, Weather and HyperUI.
You can check out their source code (master branch is for S60, maemo branch is for maemo), or you can just download S60 and/or Maemo5 Packages and try them out. There still some work needed in a place or another, but the overall status is very impressive: they run really smooth on both devices. MyBudget stills using QTimeLine instead of the Qt Animation API, but this “porting” is already in progress. The maemo packages have been compiled using Qt-X11-Maemo repository, but we also uploaded these packages for you. :)
For the lazy ones, a few screenshots:
HyperUI
Weather
ShoppingList
MyBudget
As you can see, each demo has its own conceptual idea. Our designers managed to do a great work! Patricia Montenegro, Ian Moreira and Tamara Baia made a 45″ video presenting the whole design process of the Weather demo. Patricia (patifa) also wrote a very complete post about this particular demo.
Changing the weather from Ian Moreira on Vimeo.
Let’s hope that they prepare some more nice videos about the other demos!
It was really fun to work on this. Congrats to all teams involved in the project!
so Qtie! :)
jeez
Entre 22 e 24 de Outubro estará acontecendo o Latinoware 2009, no Parque Tecnológico de Itaipu – Foz do Iguaçu.
Nós estaremos por lá! Eu e Anselmo Melo vamos ministrar um mini-curso de Qt 4.6 no sábado, 24/10. Teremos também palestras de Artur Souza, sobre o Plasma Netbook, Caio Marcelo e Eduardo Fleury falarão sobre Qt Layouts, Ana Cecília (aninha) vai falar sobre Usabilidade em OSS usando seu GSoC como estudo de caso e Kenneth vai falar sobre o Webkit.
Caso queira entrar em contato, basta nos procurar nas referidas palestras ou no stand do INdT que estará em algum lugar do evento. :)
Nos encontramos por lá!
jeez
Nokia has launched a hacking/modding contest for N900 called Push.
The main idea is based on “how would you connect something that
you love to a Nokia N900 ?”. There will be 5 winners, who will
receive funding and a N900 to make their projects become real,
and after that all projects will be displayed on several events
around Europe, on Nokia Flagship Stores. Check their FAQ for more info!
At first, the whole wouwlabs group decided to participate as a team
(a team could have up to 5 members). We made some meetings and
brainstorms and came up with several ideas. And good ideas!
But in the end, just me, annieC and calega managed to have enough
“spare” time to finish everything. On the last week we invited
Patricia Montenegro (patifa), a brazillian designer, to
enter the team and help us out!
Well, I’m very happy with the result of all submitted projects.
Below you can checkout the final documents and video for each of them:
- Pignocchio Bank: A re-design of the classical Piggy banks!
Document: Pignocchio Bank (pdf)
Video: http://www.youtube.com/watch?v=A0Zwhtf0ThA
- Sketch your World!: Automatic draws on a Etch-a-Sketch using a N900!
Document: Sketch your World! (pdf)
Video: http://www.youtube.com/watch?v=EYDyNwXqjjg
- N9P0V: A Persistence of Vision gadget using a N900 as drawing, animating and message writing tool!
Document: N9P0V (pdf)
Video: we had no time to finish this one… =/
The secret voice on the videos is from Filipe Calegario (calega). Note the well mixed brazilian+german english ascent. ;)
Hopefully we can see some these as one of the 5 Nokia PUSH winners!
And a personal thanks to all wouwlabs people and patifa for all
the efforts they put on this. You guys rock! ;)
Since the first time I saw Patifa’s presentation, I was very impressed about that Canola’s menu layout discussion. I’ve been a user of Canola since its first release, but had never thought about how hard it would be to implement such a dynamic layout.For those who don’t know or remember what I’m talking about:
The first column on the picture shows what the designers wanted from the layout’s behavior.
Here at OpenBossa@INdT, we are working on the Animated Layouts (there is a previous post, in portuguese) with the Qt Kinetic project, as you should know. On this last sprint, we decided to code some demos showing new usages of our QGraphicsLayoutProxy (QGLP), different from the previous ones, to test the new API we came up with.
And we did it.
A custom layout example, Qt Canola Layout, which demonstrates how to use QGLP “implicit” on an animated layout. Just add your widgets to this custom layout and let it handle with them! Check out the video in Openbossa channel.
We spent some days talking and brainstorming this algorithm. If you manage to check the engine’s implementation, you will see that it was a quite good work. The secret here was to find out what proportion designers were thinking of, and how to implement a ratio for that. The golden ratio was a good starting point… ;)
Maybe I’ll post about this engine again, goind deep on it. For time, don’t forget to check the latest Bossa Conference 2009 videos on our other channel, on blip.tv.
Also, check Anselmo’s post about this example and on a previous one, about the integration of Qt State Machine and Animated Layouts.
Yesterday I was at uCon Conference. In just some few words: it rocked a lot. I can’t remember last time I went to a security conference so well organized! (at least on Brazil…) The lectures, the speakers, the people, the discussions, the food and the workshops were ALL awesome! I’m really proud of being part of that. :) Not even ITA’s SSI, when it still existed, has ever managed to do such a great job! Thanks to H2HC guys and their 4 years of experience!
The conference happened on a restaurant called Jardins, at Recife.
Besides the presentations, there were 2 workshops. One was a great workshop titled “Introduction to Electronics” (leds, ir, pov, resistors, pic’s, etc) and the other one was a wouwlabs’ workshop called “Tinkering, DIY and Multitouch 101″. It was amazing and people seemed to enjoy it a lot! Actually, I wasn’t directly involved within it this time, so congratulations to annie, braga, calegario and jeraman! You all did really great! ;)
Lost it?! Don’t worry, there will be others. ;)
Life is random.
Nesse último mês começamos a trabalhar diretamente com o pessoal da Qt Software (antiga Trolltech), no projeto Kinetic.
A primeira questão que resolvemos lidar foi a transição animada de Layouts. Esse tipo de solução facilitará ainda mais o desenvolvimento de “fancy UI’s” usando o framework Qt.
O vídeo a seguir mostra um demo simples que fizemos para ilustrar melhor o que foi desenvolvido:
Nele você pode ver que temos 3 layouts (um vertical, um horizontal e um grid), aos quais vamos adicionando itens. Também fazemos resize nos layouts e temos a transição animada entre eles.
Para chegarmos nesse resultado, implementamos o QGraphicsLayoutProxy. A idéia é ter um layout proxy em cada layout da sua aplicação, mas que apontem para um mesmo widget. Quando uma mudança entre layouts ocorre, os proxies se responsabilizam por animar a transição dos widgets pelos quais são responsáveis. :)
Por enquanto a transição está “hardcoded” no LayoutProxy, mas já estamos trabalhando na versão final da api.
Tem sido um trabalho bastante divertido, principalmente porque estamos trabalhando em cima do “bleeding edge” do Qt 4.5 – que por sinal sai como LGPL agora em março – e as api’s de animação e de states (QStateMachine) está muito boa, acreditem!
Morpheuz também fez um post com um ótimo overview e Fleury postou uma excelente explicação sobre os “internals”… Inclusive com figuras bastante explicativas!
;)
No próximo dia 28/02 teremos a segunda edição da uCon Security Conference, em Recife. O evento custa a bagatela de R$ 60,00, com almoço e festa inclusos, e ocorrerá o dia inteiro no Bar e Restaurante Jardins.
Por esse preço e, principalmente, pelos palestrantes confirmados, posso garantir que vale mais a pena aparecer por lá. Além disso, no evento acontecerá um Capture the Flag e um outro desafio (que já começou), ambos com premiação. (a palestra de sniffing de gsm promete até demo real…)
Inscrições aqui: http://www.ucon-conference.org/register.php
Inscrevendo-se agora você ainda leva uma camisa!! E se for nos próximos 2 minutos uma coleção inteira de facas Ginsu e uma daquelas canetas que furava até madeira!!!
brincadeirinha…
jeez