Green's Opinion

Flutter Maps With Vector Tiles

Tue, 28 Dec 2021 12:00:00 -0800

Flutter is a huge productivity booster except for a few sharp edges, one of which is embedding maps. Maps are commonly added using the google_maps_flutter package, which uses native platform widgets to embed a Google map in your Flutter app. The resulting map looks and behaves beautifully, except when it doesn’t. The platform/Flutter combination is not perfect, and for my use-case isn’t good enough for a prodution app. This post details the issues that I encountered and how I ended up solving the problem by creaing a plug-in vector_map_tiles to add support for vector tiles to flutter_map.

There were two issues that prevented me from using Google maps in my app:

1. Scrolling Placement

My app presents a map with weather data and various charts. With mobile devices having smaller screens, scrolling is a necessity. This means that the map widget must scroll along with other content.

Flutter positions native platform widgets (platform views) over top of the Flutter view hierarchy. This clever approach to layering make them appear the same as other Flutter widgets. When platform widgets are not included in a scroll view, it works beautifully. When scrolling, the platform map widget is occasionally misplaced. For example, when scrolling vertically, the map widget is placed too high, too low, or gets stuck at the bottom of the screen.

2. Native Crash

To support creating awesome weather visualizations that can be shared on social media like this one, the app has to be able to render maps to an image. To do that, a snapshot of the map is taken and composed with other components. A bug in the Google Maps package occasionally results in a native crash when taking a snapshot. When such a crash occurs, the app is terminated - a terrible user experience.

Tackling the Issue: Failed Attempt

My first attempt to fix the issues was to create my own platform widget package to embed a platform-native Google Map. I knew that I could solve the native crash problem, and was hopeful that I could address widget positioning. After a few hours, I had something up and running, but was unable to achieve a user experience that was good enough. It was clear that any approach using platform-native widgets just wouldn’t work due to the widget positioning issue.

Flutter Native Maps

It turns out that there is a whole ecosystem of mapping alternatives to Google Maps, mostly focused on the web and built in JavaScript. One such library, Leaflet, has inspired an excellent Flutter map implementation named Fleaflet, also known as flutter_map. By embedding a map using Flutter components only, i.e. without platform views, I could avoid all of the rough edges that come with native platform components in the Flutter widget hierarchy.

flutter_map worked beautifully, but had rough edges of its own because map tiles were pre-rendered as raster images and scaled on the client. As a result, when zooming map tiles become slightly blurry, text sizes change, and theming is limited to themes supported by the service offering raster tiles.

Flutter Maps and Vector Tiles

Unlike a raster tile, which is an image, vector tiles instead represent map data as a set of points, polygons, lines and metadata with relative positioning. Clients render these vectors, resulting in sharp maps at any scale.

Another advantage of vector tiles is that map data can be rendered with a variety of themes, meaning that the color of geographic features such as land, roads and water can be any color as defined by the theme. This is especially relevant for mobile devices supporting dark mode.

For flutter_map to produce the beautiful maps that we have come to expect, vector tile support is essential.

Second Attempt: Support Vector Tiles in Flutter

As an experiment I decided to try drawing vector tiles using Flutter and Dart. How hard could it be? It turns out that drawing vector tiles is achievable. Within a few days I had rudimentary support for themes, basic land and water shapes, lines and text as a dart library vector_tile_renderer

I’m learning how to render vector tiles for maps in Dart https://t.co/YRmvpyQwJp #dartlang #maps pic.twitter.com/wxXc3sJGKu
— David Green (@dgreen) May 29, 2021

The commit history tells the story of starting with the simplest possible thing that could work, and evolving the design from there. You can see how the details of the original tile shown in the above tweet evolved too:

223 commits later, I have added support for vector map tiles to flutter_map with great results:

What’s Next

Vector tiles support in Flutter is good, but still needs work.

A major challenge is performance: Flutter needs to render each frame in less than 16ms in order to achieve 60 frames per second. Flutter uses the Skia framework to render Canvas rendering operations efficiently. Despite using Skia, which renders directly to Metal on iOS, map vector tile rendering can be slow enough to drop frames during animations.

Another area for improvement is more complete support for themes and theme styles. Currently, some theme style syntax is unsupported so it’s not possible to pick some existing themes and have them work as-is.

While there is more that can be done to further improve vector tile support in Flutter maps, it’s currently good enough that I’ve been running it in production now for several months. If you want to try it out, feel free to run the example app or check out my app Epic Ride Weather which is available on Google Play and the App Store.

maps attribution: Copyright Stadia Maps, OpenMapTiles, OpenStreetMap contributors

The post Flutter Maps With Vector Tiles originally appeared on greensopinion.com

Raspberry Pi SSH Setup

Sun, 26 Apr 2020 13:00:00 -0700

I previously blogged about the Raspberry Pi Development Flow. A key component enabling that development flow is SSH, a tool that will enable you to copy files and run remote commands on the Raspberry Pi.

Remote login to a Raspberry Pi

Following are the steps that you can take to configure SSH on your new Raspberry Pi.

If you haven’t yet got a Raspberry Pi, you’ll need to get one. I purchased mine from CanaKit. I got the starter kit, which arrived about 3 days after I placed the order.

Once you’ve got it powered on with a keyboard, mouse and monitor attached, you can follow these steps to setup SSH in a way that supports minimal fuss:

Before You Get Started
Enable SSH on the Pi
Create a New User on the Pi
Setup for Login Without a Password
Give Your Pi a Name

Background

SSH (Secure Shell) is a tool that enables you to copy files and run remote commands on your Raspberry Pi. SSH uses cryptography to ensure that your remote sessions are secure. SSH has a server portion (a daemon process) that runs on the Raspberry Pi waiting for commands, and a client portion (a command-line tool) that you can run from your development machine. With SSH you’ll be able to remotely control your Raspberry Pi, and even work on the Raspberry Pi as if you were right there in a terminal.

Before You Get Started

Note the IP address of your Raspberry Pi. To do that, follow these steps on the Raspberry Pi:

Open a terminal
Run ifconfig
Look for the section named wlan0 (if on Wi-Fi) or eth0 (if on a wired network)
Locate the line that says inet in that section, e.g. inet 192.168.1.250
Write down the 4-part IP address

For example:

$ ifconfig
eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether dc:a6:32:62:6a:63  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.250  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 2001:569:7eb7:9400:3037:daab:bcf3:2fdc  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::bec9:e792:710f:84f0  prefixlen 64  scopeid 0x20<link>
        ether dc:a6:32:62:6a:64  txqueuelen 1000  (Ethernet)
        RX packets 150  bytes 27774 (27.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 86  bytes 14509 (14.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

In the above example, the IP address from wlan0 is 192.168.1.250

Enable SSH on the Pi

To start with, you’ll need to enable the SSH daemon process on the Raspberry Pi. To do that, from the keyboard plugged into the Raspberry Pi follow these steps:

Open a terminal
Run sudo raspi-config. More details on raspi-config can be found here: raspberrypi.org raspi-config
Using the arrow keys (up/down/left/right) navigate to Interface Options -> SSH and select Yes
(optional) If you don’t want the Raspberry Pi to be running a graphical interface, it can be set to boot to a terminal instead. To do that, select Boot Options -> Desktop/CLI -> Console
Select Finish to exit the raspi-config tool
If prompted, select Yes to reboot the Raspberry Pi

Log In to the Pi

From now on, we’ll be running commands from your development machine (i.e. not the Raspberry Pi)

Login to your Raspberry Pi from your development machine using the following command: ssh pi@192.168.1.250 (where the IP address that you noted earlier is substituted)

You may be prompted as follows:

The authenticity of host 'i (192.168.1.250)' can't be established.
ECDSA key fingerprint is SHA256:22yuFgmgNxTvEnpnEogya+hpNGECgyExp3hsx5vgAi8.
Are you sure you want to continue connecting (yes/no)?

Type yes then press enter

When prompted, enter your password for the pi user on the Raspberry Pi
If successful, you should see a command prompt that looks something like: pi@raspberrypi:~ $

Create a New User on the Pi

This step is optional. To avoid having to specify a username every time we login to the Raspberry Pi, we’ll create a new user on the Raspberry Pi with the same username that we have on our development machine as follows:

Identify your username on your development machine by running whoami from a terminal. My username dgreen, which I’ll use in examples from here forward. Substitute your own in the following steps.
ssh into the Raspberry Pi (per the previous step)
Run sudo useradd -m dgreen to create the new user
Run sudo passwd dgreen to give the new user account a password

Run groups to find out which groups you should add to the new user account

pi@raspberrypi:~ $ groups
pi adm dialout cdrom sudo audio video plugdev games users input netdev gpio i2c spi

Run sudo usermod -a -G pi,adm,dialout,cdrom,sudo,audio,video,plugdev,games,users,input,netdev,gpio,i2c,spi dgreen to add the new user to the relevant groups
Run exit

To test out the new user, ssh back into the pi as follows from your development machine:

Run ssh 192.168.1.250 to ssh into the Raspberry Pi. Notice that we’re not specifying a username.
Enter a password when prompted
Do your happy dance, then run exit

To setup login without a password, we’re going to generate a public/private key pair on the development machine. If you’ve done this before, you can skip this step.

Run ssh-keygen -t rsa
Accept the default path for the file

When prompted, provide a password that will be used to encrypt the private key It should look something like this:

 $ ssh-keygen -t rsa
 Generating public/private rsa key pair.
 Enter file in which to save the key (/home/dgreen/.ssh/id_rsa):
 Created directory '/home/dgreen/.ssh'.
 Enter passphrase (empty for no passphrase):
 Enter same passphrase again:
 Your identification has been saved in /home/dgreen/.ssh/id_rsa.
 Your public key has been saved in /home/dgreen/.ssh/id_rsa.pub.
 The key fingerprint is:
 SHA256:R8BPBghF7BT6NwTLMDV6arrPw0RhPXYfeXl4usJS7QY dgreen@ahost
 The key's randomart image is:
 +---[RSA 2048]----+
 |    +BB+o. . o   |
 |    o*B+o.= + o  |
 |   .o=+o.=.+ +   |
 |    .+.. .E o    |
 |   .o . So.o .   |
 |   o.  ..oo +    |
 |  .o     . o     |
 |   oo            |
 |  ..o.           |
 +----[SHA256]-----+

Copy your public key to the Raspberry Pi as follows: ssh-copy-id 192.168.1.250
Run ssh-add to add your private key to the SSH agent running on your development machine. You’ll need to do this once before using SSH to log in to your Raspberry Pi every time you restart your development machine.

Give Your Pi a Name

Finally, give your Raspberry Pi a name so that you don’t have to memorize it’s IP address. To do that:

On your development machine, edit your /etc/hosts file and add a line that looks like this:
```
192.168.1.250 pi
```
Try it out, e.g. ssh pi

If you’re feeling really adventurous, you can configure your router with an IP reservation for the mac address on your Raspberry Pi. I’ll leave this one up to you to figure out.

Tips and Wrap Up

That’s it! From now on, your development flow can occur over SSH. You can run remote commands, copy files, restart processes, and control your Raspberry Pi in other ways from any computer on the same network as the Raspberry Pi.

Copy files using SCP, e.g. scp myproject.tgz pi:/home/dgreen
Run remote commands using SSH, e.g. ssh pi "pm2 stop myproject && rm -rf myproject && tar -xzf myproject.tgz && pm2 start myproject"
Login using ssh pi

For the optimal development flow, the next step is to automate the deployment of code and configuration to your Raspberry Pi.

The post Raspberry Pi SSH Setup originally appeared on greensopinion.com

Raspberry Pi Development Flow

Sun, 26 Apr 2020 11:30:00 -0700

As part of my grand plan to introduce my kids to programming and having fun while self-isolating, I decided to get into the world of Raspberry PI projects. While reading many of the fantastic project how-tos and tutorials that are available, I didn’t see much out there related to an optimal development flow.

Development flow for the Raspberry Pi

What is an optimal development flow? To me, it is the setup that enables you to rapidly experiment with changes.

The key here is “rapid”. An optimal development flow includes a very fast feedback loop, so that you know whether your changes are producing the desired result. If you can make 20 changes in an hour, your rate of learning will be much, much faster (for example) than 2 or 3 changes in an hour.

To enable a rate of change that fast, you’ll need to automate as much as possible to reduce the friction of deploying and trying out your changes. That way, not only will you be able to deploy and test more quickly, but the cognitive load will be greatly reduced so that your brain can stay focused on moving your project forward.

Here are the main elements that make up a great development flow:

Coding

Write code on your main development machine, i.e. not the Raspberry Pi. Yes, the Raspberry Pi 4 can run like a desktop - but it’s nowhere near as fast as a good development machine. Develop on the machine that runs fast, has a familiar keyboard and mouse/trackpad, and all of your usual development tools.

Having a develoment setup separately from the Raspberry Pi also makes it easy to finetune your development tools for multiple Raspberry Pi projects.

Code Editor or IDE

Choose a code editor or IDE that is optimal for the kind of development that you’re doing. At a minimum, it should have syntax highlighting for your language of choice, enable writing/running tests, and integrate with your source control. For me, this is usually Visual Studio Code.

Source Control

If you haven’t already, it’s worth learning how to use Git and GitHub. You’ll never have to worry about code backups again, and you’ll have a timeline of changes to your code that will make it easy to track down when a bug was introduced, or roll back to a point in time if needed.

Deployment

If you’re not writing your code on the Raspberry Pi (don’t) then you’ll need a way to get your changes there. The best way is to use SSH to copy your files there. SSH will even enable you to run commands remotely, so that you can build, run and restart your project.

Automation

Whether you’re developing on a Mac, Linux or Windows machine, make sure that you have a way to automate your deployments. For me that means having a reasonable terminal (bash or zsh) that will enable you to script your deployments. Ideally you’ll have a single script that will build, deploy and restart your project on your Raspberry PI from your development machine.

A Typical Cycle In My Development Flow

A typical cycle looks something like this:

Make changes to the code in VS Code
Write some tests to make sure that it works
Deploy to the Raspberry Pi: $ ./deploy.sh
Verify that the the changes produced the desired result on the Raspberry Pi
Commit and push to GitHub git commit -m 'some message' && git push
Rinse and repeat

Notice that there is:

no manual backup of the code
no worrying about backing up files or settings, or anything on the Raspberry Pi
no messing around copying files
no multi-step processes

Next Steps

To automate anything on your Raspberry Pi, you’ll want to have SSH setup. See Raspberry Pi SSH Setup for detailed steps describing how to do that.

The post Raspberry Pi Development Flow originally appeared on greensopinion.com

Troubleshooting Android App Crash on Chromebook

Fri, 21 Feb 2020 09:30:00 -0800

One thing is true for all developers - that is, with wide enough distribution, if a user can do something with your app, they will. If you’re building apps for Android, you will eventually have users running your app on their Chromebook. This is great for those looking for a laptop-like experience with your Android app, but what do you do when your app crashes only on Chromebook?

The official ARC developer docs have some great materials on troubleshooting, involving putting the device into “Developer mode” and connecting adb. While that may be great for your development environment, it’s clearly not practical for end users. As it turns out, there’s a much easier way. Here’s what I did:

Have the user reproduce the issue. When the app crashes, they should select the “Send feedback” option. The feedback mechanism may not complete, but that’s okay since it will save feedback to the Chromebook logs.

Next, have the user put the following in the address bar of Chrome chrome://net-internals and then select “ChromeOS”.

They can then click the “Store System and User Logs” button. After a few moments, a tarball containing logs will show up in their Downloads folder, e.g. combined-logs_20200222-081143.tar.gz Have them send that to you. In my case, the file was too big for email so I used a file sharing service (e.g. via Google Drive, Dropbox, etc.)

When you receive the tarball, extract it and open the file feedback/arc-bugreport. If you’re lucky, you’ll find a stack trace of your app crash in there. Mine looked like this:

02-22 08:04:13.732  1608  1608 I art     : Rejecting re-init on previously-failed class java.lang.Class<com.greensopinion.rideweather.activity.StatusActivity>: java.lang.LinkageError: Method boolean com.greensopinion.rideweather.activity.StatusActivity.isResumed() overrides final method in class Landroid/app/Activity; (declaration of 'com.greensopinion.rideweather.activity.StatusActivity' appears in /data/app/com.greensopinion.rideweather-1/base.apk:classes2.dex)
02-22 08:04:13.732  1608  1608 I art     :   at java.lang.Class java.lang.VMClassLoader.findLoadedClass!(java.lang.ClassLoader, java.lang.String) (VMClassLoader.java:-2)
02-22 08:04:13.732  1608  1608 I art     :   at java.lang.Class java.lang.ClassLoader.findLoadedClass(java.lang.String) (ClassLoader.java:742)
02-22 08:04:13.732  1608  1608 I art     :   at java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String, boolean) (ClassLoader.java:362)
02-22 08:04:13.732  1608  1608 I art     :   at java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String) (ClassLoader.java:312)
02-22 08:04:13.732  1608  1608 I art     :   at android.app.Activity android.app.Instrumentation.newActivity(java.lang.ClassLoader, java.lang.String, android.content.Intent) (Instrumentation.java:1078)
02-22 08:04:13.732  1608  1608 I art     :   at android.app.Activity android.app.ActivityThread.performLaunchActivity(android.app.ActivityThread$ActivityClientRecord, android.content.Intent) (ActivityThread.java:2626)
02-22 08:04:13.732  1608  1608 I art     :   at void android.app.ActivityThread.handleLaunchActivity(android.app.ActivityThread$ActivityClientRecord, android.content.Intent, java.lang.String) (ActivityThread.java:2800)
02-22 08:04:13.732  1608  1608 I art     :   at void android.app.ActivityThread.-wrap12(android.app.ActivityThread, android.app.ActivityThread$ActivityClientRecord, android.content.Intent, java.lang.String) (ActivityThread.java:-1)
02-22 08:04:13.732  1608  1608 I art     :   at void android.app.ActivityThread$H.handleMessage(android.os.Message) (ActivityThread.java:1527)
02-22 08:04:13.732  1608  1608 I art     :   at void android.os.Handler.dispatchMessage(android.os.Message) (Handler.java:102)
02-22 08:04:13.732  1608  1608 I art     :   at void android.os.Looper.loop() (Looper.java:154)
02-22 08:04:13.732  1608  1608 I art     :   at void android.app.ActivityThread.main(java.lang.String[]) (ActivityThread.java:6320)
02-22 08:04:13.732  1608  1608 I art     :   at java.lang.Object java.lang.reflect.Method.invoke!(java.lang.Object, java.lang.Object[]) (Method.java:-2)
02-22 08:04:13.732  1608  1608 I art     :   at void com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run() (ZygoteInit.java:891)
02-22 08:04:13.732  1608  1608 I art     :   at void com.android.internal.os.ZygoteInit.main(java.lang.String[]) (ZygoteInit.java:781)

You might wonder why I’d override a final method in android.app.Activity. Well, the short answer is that I made a mistake. For me the method only showed up on the Chromebook - i.e. not in the Android SDK, in the emulator or on my two Android test devices. It turns out that I should have known, see Fragment.isResumed()

With the stack trace in-hand, it’s a trivial matter to resolve the issue. Best of all, no “Developer mode” required on the Chromebook!

The post Troubleshooting Android App Crash on Chromebook originally appeared on greensopinion.com

Your Team Might Not Be As Effective As You Think

Tue, 22 Jan 2019 12:30:00 -0800

I’ve had the privilege of working with highly effective teams for most of my career. In particular, the past 5 years have been incredible, working with the best software developers that I’ve ever known. The kind of teams that books are written about — embracing software craftsmanship, self-organization, agility (small ‘a’), responsibility and continuous improvement. The kind of teams that are principled, that share knowledge freely and deliver high quality software in a repeatable, sustainable fashion. So you can imagine that it came as a shock to me, when arriving at a new company, that the new team felt better in ways that I didn’t fully understand.

My realization wasn’t instantaneous. It was a gradual, multi-day process. In hindsight, it should have been evident right away, but it was my own learning process that slowed it down. My experience was simple: in a series of meetings, discussions, and collaboration opportunities, I was working with teams of people who were not all like me. Not all white, not all male, not all with a tech background, not all the same.

Vancouver is somewhat different than many places in the world in that we have a melting pot of many different cultures and ethnic backgrounds — so ethnic and cultural diversity wasn’t new to me — but one thing really stood out at Ayogo that changed the dynamic that I was used to. In almost every situation, I was working with teams that had not just one or two women, but many women. In some cases, they comprised more than half of the team.

My realization is that these teams take a different approach to problem solving. The way that ideas, concerns and questions are put forward is different. The way that individuals are given space to add their unique value and perspective is different. I find myself wanting to sit back, waiting a few seconds here and there in the discussion to see who else will offer an opinion, leaving space for others to move in. I continue to be impressed with the unexpected directions that decisions take, going down paths that would not have occurred to me on my own.

I’ve come to realize that while individual contribution is critically important, perhaps even more important is the composition of the team. Having teams with varied backgrounds, skills, knowledge and perspective opens up all kinds of possibilities. Uniform teams are blind to these possibilities, not because they aren’t good enough, but because the nature of their composition makes them incapable of seeing. They don’t know what they don’t know.

I reached out to the team with my observations and asked for their opinions. Ayogis jumped in with enthusiasm! We now have a conversation going — around the table, on Slack, and now here. Mine is just one perspective, but my point of view has now been expanded by opening it up to everyone.

I’m having a blast at Ayogo — discovering what’s possible with teams that are unlike any other. Highly effective teams, the best that I’ve ever known, because of their diversity.

Faster Feedback Loop on Code Reviews

Wed, 13 Jun 2018 13:30:00 -0700

We’ve all seen those projects that start with fast builds. Fast builds that build everything and run all of the tests are a great way to get feedback on your code review, whether it be Travis commenting on your GitHub Pull Request or Jenkins voting on your Gerrit code review. It’s almost inevitable that successful projects end up with growing build times, often to the point where the feedback loop on our changes is out of control. This post is about solving that problem.

Traditional Approach

I’ve seen builds go from a few minutes, to 8 minutes, to 12, 15, and before you know it builds are taking more than 40 minutes. Along the way we convince ourselves that it’s OK, until eventually build times are out of control and review feedback cycle times are out the window. Keeping build times to under 20 minutes is key to any kind of efficiency, with sub-10 minute builds being the sweet spot.

Fixing the problem becomes a monumental task as the complexity of the project grows, and prioritizing that work always seems to take a back seat to more urgent feature delivery. To make it worse, as the team grows and one team becomes two or more, the responsibility of fixing the issue is not squarely within a single team and so the problem continues to go unsolved.

Our initial cut at reducing build times took some of the usual tacks:

parallel builds with better modularity
faster build machines
running fewer concurrent builds so that build machines have a lighter load
profiling/optimizing long-running integration tests

These all made a difference, but we eventually reached the limit of what we could achieve with these approaches and still, build times were reaching at best 35 minutes and in excess of 50 minutes on a bad day.

A New Take

If we can’t make our tests any faster, and we can’t make the build machines run those tests any faster, what if we just ran fewer tests? Common sense tells us that we need to run all the tests - but what if we don’t need to?

With the premise that everything on master has already passed all of the tests, then it follows that we really only need to run tests for the code that’s affected by the last change, that is, the change in our code review.

By taking the intersection of the last commit (i.e. the code review) with the transitive Maven POM dependencies and hierarchy, we can dynamically determine which projects should run tests. This is relatively trivial by using the Maven Plugin APIs and JGit (for Git repositories).

To run this experiment, I created the maven-change-impact Maven plug-in. Details are on the project page on GitHub, but to summarize: the maven-change-impact plug-in sets a Maven property based on analysis of the Git change and the POMs. This property can then be used to enable or skip execution of tests using the Maven Surefire Plugin on a per-project basis.

Results

We’ve been running with this experiment now for two weeks and we’ve seen impressive results.

On average, our builds have come down to 12 minutes. Some builds are running as fast as 4 or 5 minutes, while others are in the 18-20 minute range. This is a massive improvement over our previous average which was in the range of 40-50 minutes.

Side-Effects

With faster builds, we have the initial benefit of faster feedback cycle time which translates to keeping developers in the flow.

But, there are other benefits too:

With faster builds, we’ve seen an increase in the number of review verification builds that we’re running. In other words, we have better throughput. This translates to being able to move more changes through our value stream, faster. In other words, more value to the customer and improved ability to react to change.

Developers are no longer incentivized to create large reviews. More but smaller changes reduces the cognitive overhead in code reviews, and makes for better separation of unrelated changes. So developers are able to collaborate more effectively, and it’s easier to understand the scope of any given change.

Applicability

This approach will benefit teams the most in cases where build times are long primarily due to test execution times, and you have reasonably good modularity in your codebase.

For example, if you want to run integration tests in addition to unit tests to verify code reviews and your integration tests take more than a few seconds each, test execution times can really add up. This definitely applies in our case, where we have more than 30,000 tests.

Try It Out

I invite you to try out the maven-change-impact to see if it makes a difference for you and your team. Enjoy, and don’t hesitate to let me know if it works out for you!

The post Faster Feedback Loop on Code Reviews originally appeared on greensopinion.com

Surfacing Abstractions in Code Reviews

Fri, 27 Apr 2018 14:30:00 -0700

Code review tools such as GitHub PRs and Gerrit do a a great job of enabling developers to collaborate, but their file- and line-oriented approach leaves abstractions out of the picture. Developers must have the capacity to see the abstractions by reading the code, or risk having code reviews devolve into a critique of syntax, whitespace and implementation detail. The intrinsic ability to see abstractions comes easily for some, but others are at a disadvantage especially when less familiar with the codebase. To address this issue, I conducted an experiment in surfacing code abstractions as a diagram in code reviews.

My premise was that a static class diagram would enable developers to see the major abstractions and their relationships independently of the implementation detail. By having this higher level view of the code related to a change, developers would have a deeper understanding when performing a code review. My goal was two-fold: to enable ramp-up of less experienced engineers, and to draw more attention to the abstractions since in many cases they as if not more important than the implementation. (A poor abstraction implemented perfectly is in many cases less valuable than a great abstraction with a sub-optimal implementation.)

To run this experiment, I created a plug-in for Gerrit that dynamically generates UML static class diagrams for any code review. These diagrams are intentionally missing detail, taking the UML-as-sketch approach to avoid overly-complicated diagrams.

I was looking for answers to the following questions in this experiment:

Could diagrams be automatically generated that were good enough to communicate the major abstractions?
Is it possible to pull in more of the surrounding context so that the changed code can be evaluated within the context of a larger codebase?
Would developers find these diagrams useful?
Would these diagrams affect the code review process sufficiently to driver higher quality changes to non-trivial systems?

The result of this experiment was surprising. A simple, well-structured code review produced the following diagram:

This is what it looks like on the Gerrit code review:

So far, so good. But, what about a more complicated example? Following is a diagram generated from a real contribution:

This diagram is much harder to read, even when zoomed in. My initial reaction was that this diagram was not useful at all. The code and its surrounding context were simply too complicated to enable generation of a useful diagram. After saying that to myself, it dawned on me: This is exactly why the diagram is useful. The generated diagram is exposing code for what it is: overly complicated with poor abstractions and too many dependencies.

In my experience, most developers aren’t as practiced at reading static class diagrams as they are at reading code. For these diagrams to be useful, developers would need to become proficient at reading them. Proficiency comes with practice, but for that to happen, developers would have to be willing to try out the approach.

This experiment isn’t over. The next step is to try it out for a few weeks and get feedback from developers.
What do you think, would this be helpful for you or your team? If you’re interested to try it out, the project has been published on GitHub.

The post Surfacing Abstractions in Code Reviews originally appeared on greensopinion.com

Architecting for the Value Stream

Mon, 22 Jan 2018 12:00:00 -0800

More features, faster time to market, fewer defects. These are what organizations strive for. But an overly narrow continuous integration and continuous delivery focus on DevOps is insufficient to achieve these goals. Instead, we should be taking a whole-system point of view that considers factors beyond development and operations — including people, teams, processes and tools. By taking a constraints perspective, we can identify and address flow-limiting aspects of our value stream.

The foundation of “good” architecture — principles such as modularity, separation of concerns and SOLID — deliver the promise of flexibility, maintainability and all of the other “ilities” that we value. But these qualities, while enabling faster and sustainable delivery of business value, fail on their own to address some constraints that limit the value stream. For example, delivery aspects such as multiple teams; new team member ramp-up; support and troubleshooting; and user feedback impose constraints on our value stream. By taking a wider perspective than only technically driven architecture aspects, we can optimize it to remove some constraints.

Collaboration with multiple delivery teams

Multiple delivery teams can deliver more features faster than one, but this only holds true if introducing more teams doesn’t introduce additional constraints. But multiple teams introduce greater potential for conflicts and corresponding churn as more people work on the same codebase, build pipeline, tests and infrastructure. So how do we minimize this problem with our architecture?

With a technology-first approach, you may architect your module or microservice boundaries in a way that optimizes separation of concerns and cohesion, e.g. loosely coupled functionality grouped by related concepts. Meanwhile, architecting value stream first considers the alignment of teams and value streams to the technology. As such, you may create API, microservice and module boundaries that provide autonomy to teams, enabling them to deliver features independently of other teams. And you will plan for evolving those APIs to avoid wait states that can lead to slower cycle times.

For example, if a team must wait four weeks for a new API that is required for a new feature, you would refine your architectural process to remove the dependency, provide an accelerated way of having that team contribute to the API, or take the architectural debt of copying the functionality while ensuring you have a path to remove that debt.

Let’s take this thinking and apply it to other aspects of our delivery: build and test infrastructure. Is one team causing build breakages for another? While modifying a test fixture configuration, does that team cause another’s tests to fail? Our natural tendencies might be to share build and test environments to avoid duplication and minimize redundancy. If you are architecting these systems value stream-first, you will design the build and test pipeline and corresponding environments in a way that insulates each team. Meanwhile, a share-nothing approach, while it may introduce redundancy, could be a great trade-off to eliminate dependencies and friction between teams.

One way of achieving share-nothing builds is to have tests provision their own environments for the required duration. For example, a test suite that verifies services running over MySQL could bring up a containerized MySQL instance for the duration of the test, and destroy it when the test is complete. Effort is required to enable tests to accomplish this level of provisioning; however once complete the burden of creating, maintaining and administering long-running test fixtures is eliminated.

Feedback loop

A good feedback loop is an essential part of enabling the value stream. Tests provide feedback to engineers so that they know whether the system works. Fast tests, running earlier in the process (for example in the IDE) create a shorter feedback loop, enabling engineers to react more easily and more quickly. A great feedback loop, however, goes beyond test speed and the trade-off between unit tests and integration tests per the Test Pyramid. Test speed is important, but only covers a very small part of the engineering delivery cycle.

The biggest constraint on receiving user feedback is typically the user. To remove this constraint, you must remove the user, which is a paradox since user feedback is what we seek. While it may not be possible to remove the user for all user feedback, we can for a class of it. By architecting a product that automatically provides feedback without the user having to instigate it, we can approximate user feedback and eliminate the constraint.

For example, you should build metrics into the application so that you know before the user if a new feature is too slow. Monitoring, instrumentation, telemetry and analytics are essential feedback tools that should be integrated into every application, not just mobile apps.

Cycle time and sustainability

In many organizations, focusing on the value stream means delivering new user features. Often, it’s seen as a business decision as to whether to deliver a new feature and possibly incur additional technical debt or pay off existing technical debt. It is indeed a business decision, but making any decision should include understanding the consequences of that choice.

The true cost of technical debt is often misunderstood. By definition it slows down you, your team or the organization. While one small technical debt may not impact you significantly, pile up a few of them and your velocity will suffer. Pile up more than a few of them, and beyond slower delivery speeds, team morale and team culture will suffer.

The choice of whether to accumulate or to address new technical debt is often made without understanding the scope and quantity of existing technical debt. Take great care to avoid a pile-up. Instead err toward having little or no technical debt at all. While it may sound like an unattainable goal, it is worth pursuing, since it enables teams to perform at their highest level.

New team member ramp-up

Staff turnover is a normal part of software delivery. Consider the time required to make new staff productive, the mistakes they make while ramping up and their consequences. These strategies can help on-boarding issues:

Avoid documenting, instead automate: Automated things don’t require learning to get them right, and if done well are self-documenting. Automated things run the same way every time, don’t require learning how and tend to run very fast. Automating everything is a great strategy for reducing the on-boarding burden.
Modular systems are easier to learn than monolithic systems because of their decoupled nature. Components have fewer dependencies, and are grouped by functionality. This means that new team members have less to learn in specific areas, since they don’t need to understand the whole system. This translates to faster time-to-value.

Thinking holistically about the value stream as it relates to the whole organization can provide insights into improved software architectures. A traditional architecture view is limited to the technical merit of a solution; however you may need a broader view to address the competitive needs of modern organizations by delivering a steady stream of value to customers. Seek architectural strategies to address issues that involve more than just the technical aspects of software delivery. Consider too, the teams and people that are involved from idea to delivery and operations.

ELK Stack for Improved Support

Sat, 09 Sep 2017 16:11:00 -0700

The ELK stack, composed of Elasticsearch, Logstash and Kibana, is world-class dashboarding for real-time monitoring of server environments, enabling sophisticated analysis and troubleshooting. Could we also leverage this great tooling to our advantage in situations where access to the server environment is an impossibility? Recently while investigating a customer support case, I looked into whether or not we could create a repeatable process to enable analysis of log files provided as part of a support case. Following are details of the approach that we came up with.

The requirements were pretty straight-forward:

Enable analysis of multi-GB logs provided as part of a support request
Use familiar, first-class tooling
Zero-installation usage by anyone on the support or engineering team
Zero maintenance (no infrastructure needed)

For this we chose ELK and Docker Compose, with the idea that anyone could bring up and tear down an environment with very little effort. Rather than monitor logs in real time however, we needed to pull in logs from a folder on the local machine. For this we used Filebeat.

This is the docker-compose.yml that we came up with:

elk:
  image: sebp/elk
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"
  volumes:
    - ${PWD}/02-beats-input.conf:/etc/logstash/conf.d/02-beats-input.conf
    - ${PWD}/log:/mnt/log
filebeat:
  image: docker.elastic.co/beats/filebeat:5.5.1
  links:
    - "elk:logstash"
  volumes:
    - ${PWD}/filebeat.yml:/usr/share/filebeat/filebeat.yml
    - ${PWD}/log:/mnt/log

This Docker Compose file brings up two containers: elk, which as you might have guessed runs Elasticsearch, Logstash and Kibana, and filebeat, a container for reading log files that feeds the elk container with data.

The filebbeat container is the most interesting one: it reads files from a local folder named log in the current directory of the Docker host machine. With the brilliance of ${PWD} support in Docker Compose, all we have to do is move support log files into that folder!

The following filebeat.yml configuration is needed:

filebeat.prospectors:
- input_type: log
  paths:
    - /mnt/log/*
  include_lines: [".*? ERROR "]
  multiline.pattern: ^\s*\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d \[
  multiline.negate: true
  multiline.match: after

processors:
- add_cloud_metadata:

output.logstash:
  # The Logstash hosts
  hosts: ["logstash:5044"]

This one is configured to handle multi-line log entries (including Java stack traces) where the initial line of each log entry starts with a timestmap. The multiline.pattern above may need adjusting to suit your log files.

All that remains to get this working is the beats configuration, 02-beats-input.conf, which uses a bit of filtering hackery to split up the unstructured log entries into structured data before it’s added to Elasticsearch:

input {
  beats {
    port => 5044
  }
}
filter {
  grok {
    match => {
      "message" => "\s*(?<entry_date>\d\d\d\d-\d\d-\d\d) (?<entry_time>\d\d:\d\d:\d\d),(?<entry_time_millis>\d\d\d) \[(?<thread_id>[^\]]+)\] (?<severity>[^\s]+) (?<category>[^\s]+) - (?:(?<error_code>CCRRTT-\d+(E|W)):\s+)?(?<message_text>.*)"
    }
  }
  mutate {
    add_field => {
      "entry_timestamp" => "%{entry_date}T%{entry_time}.%{entry_time_millis}Z"
    }
    remove_field => ["entry_date", "entry_time", "entry_time_millis"]
  }
  mutate {
    remove_field => ["message"]
  }
  mutate {
    add_field => {
      "message" => "%{message_text}"
    }
    remove_field => ["message_text"]
  }
  grok {
    match => {
      "message" => "\s*(?<message_summary>.*?) Cause Context:.*"
    }
  }
  grok {
    match => {
      "message_summary" => "\s*(?<message_first_sentence>.*?\.).*"
    }
  }
}

After creating those files I ended up with the following:

./
./docker-compose.yml
./logs/
./02-beats-input.conf
./filebeat.yml

With a simple docker-compose up, I moved over 56GB of log files into the logs folder and grabbed coffee. After a few minutes I was happily analyzing the situation using a Kibana dashboard:

In this example, we can see a chart of error codes and distinct messages over time.

To make this process even smoother, we used elasticdump to export our Kibana dashboards for other support cases.

To export dashboards:

elasticdump --input=http://localhost:9200/.kibana --output=$ --type=data > kibana-settings.json

To import dashboards:

elasticdump --input=./kibana-settings.json --output=http://localhost:9200/.kibana --type=data

Using ELK for post-mortem analysis of log files is a snap. The approach outlined above makes the process repeatable with trivial steps that anyone can follow, with no need to maintain ELK infrastructure.

The post ELK Stack for Improved Support originally appeared on greensopinion.com

Using Eclipse for Android Development in 2017

Sun, 23 Apr 2017 13:05:00 -0700

Innovation in platforms and tooling is necessary for companies to remain competitive, and Android development is no exception. The Android team at Google has been working hard in this department, with their latest major new thing being improved Java 8 support. This involves some big changes to the Android build system, which unfortunately has caused some casualties in tooling such as this previous approach to using Eclipse for Android development.

With the prospect of cleaning up my Android codebase with Java 8 lambdas, I was unable to resist the new Java 8 tooling. A suggestion from Johannes Brodwall got me thinking about a Gradle plug-in to simplify generating an Eclipse .classpath in a way that is compatible with the innovation going on in Android tooling. Feeling inspired, I created this plugin which does the heavy lifting, adding the necessary classpath entries and expanding Android Archive (AAR) files so that their nested jars can be included.

To use the plugin, add the following to your build.gradle:

apply plugin: 'com.greensopinion.gradle-android-eclipse'
apply plugin: 'eclipse'

buildscript {
    repositories {
        maven {
          url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
      classpath "gradle.plugin.com.greensopinion.gradle-android-eclipse:android-eclipse:1.0"
    }
}

eclipse {
  classpath {
    plusConfigurations += [ configurations.compile, configurations.testCompile ]
    downloadSources = true
  }
}

Then from the command-line run:

$ gradle eclipse

When done, a .classpath and .project file should be in the current folder. You’ll need to run gradle eclipse whenever changes are made to your classpath.

To find out more about the plugin, head on over to https://github.com/greensopinion/gradle-android-eclipse

Enjoy!

The post Using Eclipse for Android Development in 2017 originally appeared on greensopinion.com

This In-App Purchase Has Aready Been Bought

Wed, 22 Mar 2017 16:14:00 -0700

If you’re developing iOS apps with in-app purchases, chances are you’ve seen this dreaded dialog and wondered why SKPaymentQueue isn’t calling your SKPaymentTransactionObserver.

“This In-App Purchase has already been bought. It will be restored for free.”

This scenario can easily occur when an error occurs before calling SKPaymentQueue.default().finishTransaction(transaction).

Apple has designed the payment queue in such a way as to enable apps to fulfill a purchase before completing - with finishTransaction being the indication that fulfillment is complete. In the case that a failure occurs during fulfillment, the app can recover when SKPaymentQueue calls the app’s observer again.

This is all good and fine - except many developers have encountered a situation where this process breaks down. Their SKPaymentTransactionObserver is never called again, and the user is presented with the dreaded “already bought” dialog and no apparent way to fix the problem.

Like many problems in software development, it turns out that the problem is quite simple and completely non-obvious. SKPaymentQueue does indeed call a transaction observer - just not the one in your app. A library (in my case Firebase analytics) is adding its own SKPaymentTransactionObserver before your app adds its own observer. As a result, SKPaymentQueue is calling that observer first, and by the time your observer is added, as far as the payment queue is concerned it has already delivered the notification and there’s nothing left to do.

To fix the problem, ensure that your app registers its own observer before any third party libraries get a chance to do the same:

    class AppDelegate: UIApplicationDelegate {

        func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {

            ...

            // ORDER DEPENDENCY: before analytics
            SKPaymentQueue.default().add(createPaymentTransactionObserver())

            // ORDER DEPENDENCY: do this after store service setup!
            FIRApp.configure()

            return true
        }
        ...
    }

That’s all!

The post This In-App Purchase Has Aready Been Bought originally appeared on greensopinion.com

Mylyn WikiText on Maven Central

Mon, 06 Mar 2017 12:00:00 -0800

I’m excited to announce that for the first time, Mylyn WikiText is now available on Maven Central.

This release enables use of Mylyn WikiText within a normal Maven project; previously Mylyn WikiText could only readily be consumed by downloading the jar files or by resolving dependencies using Tycho.

Now dependencies can be resolved at the following coordinates:

<dependencies>
    <dependency>
        <groupId>org.eclipse.mylyn.docs</groupId>
        <artifactId>org.eclipse.mylyn.wikitext</artifactId>
        <version>3.0.2</version>
    </dependency>
    ...
</dependencies>

Mylyn WikiText provides several additional components (e.g. for Markdown, AsciiDoc, etc.) which are listed here.

To celebrate the new release I’ve created an examples project on GitHub, which shows how to use Mylyn WikiText in a few different ways:

https://github.com/greensopinion/wikitext-examples

The examples demonstrate:

Mylyn WikiText Java APIs
Mylyn WikiText Maven mojo to generate HTML help using a pom
Mylyn WikiText Ant tasks from within Maven

From now on Mylyn WikiText releases will provide Maven artifacts from Maven Central, and p2 artifacts for OSGi and Eclipse applications.

Enjoy!

The post Mylyn WikiText on Maven Central originally appeared on greensopinion.com

Building a Cross-Platform Desktop Application with AngularJS and Java

Wed, 01 Jun 2016 13:00:00 -0700

Building cross-platform desktop applications usually means compromising on user experience, ease of development or both. Below I detail the reasoning and trade-offs behind an approach that involves state of the art technologies and development methodology with fewer compromises.

Web technologies (HTML, CSS and JavaScript) are arguably the best choice for a cross-platform UI in most situations. With numerous great choices for frameworks, skills commonly available, and unparalleled flexibility in styling, it’s hard to imagine choosing anything else as long as you don’t mind your app looking like a web application.

Creating applications with the likes of Electron or MacGap is nothing new - there are several successful such applications in widespread use, such as Slack, VisualStudio Code and WordPress Desktop. But building a JavaScript-only application isn’t for everyone, especially in cases where reuse of existing functionality is important.

Many organizations have substantial functionality available in the form of Java libraries that they would like to reuse and the corresponding skills, infrastructure and methods to build and maintain application logic in Java. Taking an Electron-like approach to building such applications but with an emphasis on Java for the functional portion of the application could be a good choice in these cases.

Java ships with a WebKit-based WebView for JavaFX. This made a bit of a splash, as previous attempts to embed a browser in Java apps were either using a sub-par Java-based browser, or had crazy dependencies and complex setup.

Creating a JavaScript application using a JavaFX WebView has been done before. On the surface this approach looks great - with the benefits of a state-of-the-art browser engine, standard APIs, no extra dependencies and tools to create platform-specific installers, many of the previous problems are solved. But the approach hasn’t seen wide adoption, in part due to these shortcomings:

1. HTML and CSS Tooling

Web developers rely heavily on browser-based tools to develop the look and feel of an application. Modern web browsers provide integrated tooling that provide a way to introspect and modify CSS and HTML. With a couple of clicks, developers can troubleshoot and try out styles and HTML, live in the application. This ability to change an application as it’s running is one of the cornerstones behind web developer productivity.

A JavaFX WebView doesn’t have such tooling. The WebView is a black-box, essentially locking developers out from their most powerful development approach. It’s like going back to the mid ’90s.

One approach to overcoming this limitation is to include Firebug Lite, which enables inspection of HMTL and CSS in almost any browser, including the JavaFX WebView. I did get it working, and though it’s better than not having any tooling, not all versions of Firebug Lite worked for me and I was unable to edit CSS properties live in the application. To be fair, Firebug Lite is not intended as a replacement for Firebug or Chrome development tools. In the end I found that it was insufficient for my needs.

2. JavaScript Debugging

No matter how good your JavaScript automated testing approach, there is no substitute for debugging JavaScript live in a running application. It’s the last-resort fall-back to figuring out what’s really happening in your application. With no built-in JavaScript debugger, the JavaFX WebView left me feeling like it was the early days of the web.

3. JavaScript to Java and Back Again

With the UI in a WebView, calling out to your Java application (or the reverse, calling into the JavaScript web UI) is an important aspect of this approach. Unfortunately crossing the JavaScript-Java boundary is not entirely straight-forward. Though it’s possible to invoke JavaScript in the WebView from Java, and call back to Java from JavaScript, it’s not transparent.

Data type conversion is very limited; most non-trivial use cases will require Java code that introspects JSObject, a generic JavaScript type wrapper. This makes for clumsy Java code and in most cases reduces the Java-to-JavaScript boundary to a message-passing paradigm.

4. WebKit Version

Having a WebKit-based browser ship with the Java platform is great, but which version is it? It appears that updating WebKit on the Java platform is quite a bit of work, since Oracle is not updating the WebKit version very frequently. As of this writing, Java 8 ships with WebKit version 537.44, which is from 2013.

If the embedded WebKit is used as a web browser - that is, to browse the web, not just for your application - then having an up-to-date WebKit is critical for security.

5. Download Size

Distributing a JRE with the application is needed to ensure that the application runs consistently and to keep installation simple (avoiding the need to install a JRE separately). As a result the application size is over 100MB, which could be problematic for some situations.

Mitigating the Drawbacks

With a laundry list of drawbacks our approach of using a JavaFX WebView to host the UI for a desktop Java application is sounding pretty bad so far. But, with a concrete mitigation strategy for each issue, it could be worth exploring further:

Web Development Approach

Web development tooling leverages our team’s expertise and is key to productivity - so how can we come up with a development approach that enables this?

It turns out that we can build a traditional web application by implementing REST services as the interface between our UI and our application. By having the UI call out to REST services, we can have our cake and eat it too.

With this approach our application architecture looks something like this:

In development we can deploy the application as a normal web application like this:

By providing an in-memory HTTP/REST bridge to these services we can run the UI and application logic in one process, while in development we can split the application from the UI by using a web browser and web container. This means that we get the single-process deployment profile of a desktop application, while retaining the web development tools and techniques that we value so much.

Java to JavaScript Impedance Mismatch

To overcome the difficulty of type compatibility and JSObject called out above, we turn to our REST services again. JavaScript with frameworks like AngularJS is purpose-built to call REST services. And, by passing JSON data over HTTP (or over the REST bridge) all of our data is serialized to a standard format (JSON) which is trivial to deserialize on the Java end. In fact, the impedance mismatch completely disappears by implementing REST services and it feels like developing a “normal” web application.

Build System

A major problem for any project is the build system. Maven, Ant, Gradle - whatever you choose, we still need something to build the webby front-end. For this application I chose Grunt with Node, Karma, Jasmine… it’s all there. Ultimately this is the same as any web application using more than one implementation language, there are two build technologies involved, which complicates things significantly.

Testing Strengths

Web technologies for testing the UI are fairly mature and enable testing on several levels:

Unit testing is pretty straight-forward when using AngularJS, Karma and Jasmine, enabling a high level test coverage for JavaScript backing the UI with a reasonable effort.
Karma and Jasmine can also be used to test for elements in the DOM.
Though I haven’t done it here, with use of Selenium WebDriver it’s possible to test application functionality via the UI.
Using a Java back-end, it’s really simple to use normal unit testing techniques and frameworks, for example with JUnit and Mockito.

Selecting web technologies for the UI makes thorough testing a possibility, covering all aspects of our application architecture.

The only real problem here is that our UI tests will be running using a web browser rather than the JavaFX WebView. It’s hard to say if that’s a major gap or if it’s something that we could live with - I think only experience would shed light in that area.

Migration to Web

If future application deployment as a real web application rather than a desktop application is a consideration, then this approach could be a great way to move forward. The application architecture being a traditional web stack lends itself very well to a web deployment.

Working Example

A working example of such an application is published on GitHub. I’d love to hear your feedback and thoughts on the application and the approach.

Summary

Building a desktop application using a JavaFX WebView with an embedded AngularJS app is not simple to set up, but does enable reuse of existing Java services and business logic. For teams that are want to leverage existing skills (e.g. Java and HTML/JavaScript/CSS) this could be a good choice, as long as having a desktop application look like a web application is OK.

Many, but not all of the development difficulties introduced by running a web application in a JavaFX WebView instead of a standard web browser can be mitigated by supporting a normal web development flow as outlined above.

For me this was a great learning experience and potentially worthy of further exploration. Would I use this as a product development approach for a real application? Maybe, if the trade-offs were to weigh in favour for the needs of the organization and application under consideration. But I’m not completely sold on it - so would keep my eye open for signs of an unhealthy development process and architecture.

The post Building a Cross-Platform Desktop Application with AngularJS and Java originally appeared on greensopinion.com

Firebase Mobile Analytics First Experiences

Wed, 25 May 2016 13:00:00 -0700

More information is better when it comes to improving any application. With a mission to find out how people are using a new Android app that I’m working on, I set out to integrate analytics into my mobile app. Imagine my excitement to discover that Google has launched Firebase Analytics for mobile apps at Google I/O 2016. It took me all of about 20 minutes to get the initial analytics integration working. Here’s what I did:

1. Setup a Firebase Account

The first step is to create a Firebase account. This was really simple using my Google credentials. Firebase was able to create a new analytics project for my existing mobile application project. To do that I selected “Import Google Project” and provided the package name (found in AndroidManifest.xml), and the SHA1 certificate fingerprint from my signing digital cert. The fingerprint can be obtained with a command like this one:

 $ keytool -list -v -keystore ~/my-keystore.jks
 ...snip..

 Alias name: alias_name
 Creation date: 25-May-2016
 Entry type: PrivateKeyEntry
 Certificate chain length: 1
 Certificate[1]:
 Owner: CN=David Green, OU=Unknown, O=David Green, L=Vancouver, ST=British Columbia, C=CA
 Issuer: CN=David Green, OU=Unknown, O=David Green, L=Vancouver, ST=British Columbia, C=CA
 Serial number: 60eef983
 Valid from: Wed May 25 21:02:06 PDT 2016 until: Sun Oct 11 21:02:06 PDT 2043
 Certificate fingerprints:
 	 MD5:  24:4C:92:9B:0D:01:26:E0:1A:49:DF:01:8F:3B:63:A0
 	 SHA1: 5B:16:23:8A:A0:30:8D:B6:B6:7F:7B:96:EE:F3:10:6C:95:EF:0D:9F
 	 SHA256: 2E:0E:29:D2:67:9D:12:65:46:C4:F5:46:A0:8B:D8:8F:FE:3C:A9:C5:6B:E4:A2:8E:FA:01:18:3E:6A:82:AE:37
 	 Signature algorithm name: SHA256withRSA
 	 Version: 3

2. Add Firebase Dependencies

Next was adding Firebase dependencies to the build. The instructions are great, it took all of 2 minutes. The only hiccup is that I had to update an existing project dependency com.google.android.gms:play-services-maps to the 9.0.0 version to get the build to work.

3. Add Analytics To The App

Finally adding analytics to the app was trivial. A key step for me is that this feature has to be opt-in. In other words, the user has to agree to providing stats, otherwise it just doesn’t feel right. To make it easy to manage the opt-in setting I wrapped the Firebase APIs in a small class called Analytics as follows:

public class Analytics {

	private static final String CONTENT_TYPE_ACTIVITY = "activity";

	static final String ENABLED_KEY = "enableAnalytics";

	private boolean enabled;
	private FirebaseAnalytics firebase;

	public Analytics(Context context) {
		checkNotNull(context);
		this.enabled = PreferenceManager.getDefaultSharedPreferences(context).getBoolean(ENABLED_KEY, false);
		if (enabled) {
			firebase = FirebaseAnalytics.getInstance(context);
		}
	}

	public void logActivated(Activity activity) {
		if (!enabled) {
			return;
		}
		Bundle bundle = new Bundle();
		bundle.putString(FirebaseAnalytics.Param.ITEM_ID, activity.getClass().getSimpleName());
		bundle.putString(FirebaseAnalytics.Param.CONTENT_TYPE, CONTENT_TYPE_ACTIVITY);
		firebase.logEvent(FirebaseAnalytics.Event.SELECT_CONTENT, bundle);
	}
}

To make this work with all of the activities in my app, I created an Activity base class as follows:

public class AnalyticsActivity extends AppCompatActivity {
	protected Analytics analytics;

	@Override
	protected void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);
		analytics = new Analytics(this);
	}

	@Override
	protected void onResume() {
		super.onResume();
		analytics.logActivated(this);
	}
}

Finally with a little search-and-replace, I had all activity classes extending AnalyticsActivity instead of AppCompatActivity.

That’s it! With just this mobile analytics will help me to understand how much and how often each activity within the app is used in much the same way as Google Analytics does the same for the web. Now only if creating a privacy policy was as straight forward!

The post Firebase Mobile Analytics First Experiences originally appeared on greensopinion.com

Using Eclipse for Android Development in 2016

Sun, 15 May 2016 13:00:00 -0700

Using the best tool for the job has a direct impact to my overall sense of well-being and happiness - not to mention productivity. For me when it comes to Java, that means using Eclipse. Life is just too short for messing around. Eclipse has been relegated to the back benches for Android development now that Android Studio is based on IntelliJ. So what is an Eclipse die-hard to do?

I found that learning Android Studio was pretty straight-forward. I was up and running, creating a reasonable Android app within a couple of days, forcing myself to learn the “IntelliJ-way”, key bindings and all. After a couple of weeks rolled by, then months, I realized that my creativity was being hampered because I just couldn’t hit the flow in IntelliJ. The friction was affecting me: my code had fewer unit tests than I would like, and simple tasks were taking too long. IntelliJ was just slowing me down too much and it was impacting my project.

Seeing that Gradle buildship doesn’t yet support Android projects, A quick google for "Gradle generate Eclipse classpath" and I landed on this StackOverflow. With a few Gradle edits to build.gradle, I was able to generate the Eclipse .classpath and .project files needed to start editing Java source in Eclipse. My configuration is based on the answer provided by Johannes Brodwall:

Update for 2017, the approach outlined here is now available as a Gradle plugin: see Using Eclipse for Android Development in 2017

apply plugin: 'eclipse'

eclipse {
// pathVariables 'GRADLE_HOME': gradle.gradleUserHomeDir, "ANDROID_HOME": android.sdkDirectory
    classpath {
        plusConfigurations += [ configurations.compile, configurations.testCompile ]

        file {
            beforeMerged { classpath ->
                classpath.entries.add(new org.gradle.plugins.ide.eclipse.model.SourceFolder("src/main/java", "bin"))
                // Hardcoded to use debug configuration
                classpath.entries.add(new org.gradle.plugins.ide.eclipse.model.SourceFolder("build/generated/source/r/debug", "bin"))
                classpath.entries.add(new org.gradle.plugins.ide.eclipse.model.SourceFolder("build/generated/source/buildConfig/debug", "bin"))
                classpath.entries.add(new org.gradle.plugins.ide.eclipse.model.SourceFolder("build/generated/source/aidl/debug", "bin"))
            }

            whenMerged { classpath ->
                def aars = []
                classpath.entries.each { dep ->
                    if (dep.path.toString().endsWith(".aar")) {
                        def explodedDir = new File(projectDir, "build/intermediates/exploded-aar/" + dep.moduleVersion.group + "/" + dep.moduleVersion.name + "/" + dep.moduleVersion.version + "/jars/")
                        if (explodedDir.exists()) {
                            explodedDir.eachFileRecurse(groovy.io.FileType.FILES) {
                                if (it.getName().endsWith("jar")) {
                                    def aarJar = new org.gradle.plugins.ide.eclipse.model.Library(fileReferenceFactory.fromFile(it))
                                    aarJar.sourcePath = dep.sourcePath
                                    aars.add(aarJar)
                                }
                            }
                        } else {
                            println "Warning: Missing " + explodedDir
                        }
                    }
                }
                classpath.entries.removeAll { it.path.endsWith(".aar") }
                classpath.entries.addAll(aars)

                def androidJar = new org.gradle.plugins.ide.eclipse.model.Variable(
                    fileReferenceFactory.fromPath("ANDROID_HOME/platforms/" + android.compileSdkVersion + "/android.jar"))
                androidJar.sourcePath = fileReferenceFactory.fromPath("ANDROID_HOME/sources/" + android.compileSdkVersion)
                classpath.entries.add(androidJar)
            }
        }
    }
}

eclipseClasspath.dependsOn "generateDebugSources"

The main difference between my configuration and that provided on the StackOverflow is the addition of the aidl path.

To get this going the first time, or to update it whenever the project configuration changes, simply run the command:

$ gradle eclipse

Or, if using the AndroidStudio-generated gradlew:

$ ../gradlew eclipse

Within a couple of minutes I had the Java imports organized and compiler warnings gone, was refactoring in anger and writing unit tests with one of my favourite Eclipse plug-ins, MoreUnit.

The post Using Eclipse for Android Development in 2016 originally appeared on greensopinion.com