I’m moving.
I’m moving blogging platforms.
My new blog is at tianjara.net/blog.
The new RSS feed is at http://tianjara.net/blog/index.rss.
The reason for this switch is,
- It is much easier for me to write new blog posts on the new system. Just a vim and make is all that is needed. No need to load up the clunky wordrpess.com which loads non-free code and pulls code from up to 7 different domains.
- I can keep my blog in git and sign my blog posts with GPG.
- I have full control over my blog (no longer at the mercy of wordpress.com)
- I can use the single domain name I’ve taken up for my digital home
- I can avoid unwanted ads placed by wordpress.com
Dilemma of the day: remap capslock as backspace or escape?
The dilemma of the day.
Should I remap capslock as backspace or escape?
Escape pros: makes mode switching in Vim really easy.
Escape cons: I don’t really use Esc much, except in vim. Perhaps there are apps where accidentally hitting esc (much easier when it is remapped to the capslock key) could be dangerous?
Escape neutrals: I don’t really need a handy esc for vim mode switching, because you could use Ctrl+C in Vim. That said, capslock is easier than Ctrl+C.
Backspace pros: I make a lot of typing errors, having a closer backspace would be a real help.
Backspace neutrals: Perhaps auto-corrective features would eliminate the need to backspace to manually correct typos.
Verdict. I think I’ll go with mapping it as escape.
Cartograhpic Principles
I’ve recently read an article which has given me a huge insight into some cartographic principles. Principles which I had been using were given the rigour they deserved.
What I gain most from it was that one kind of map among many is the physical map. The physical map is designed to portray the feeling of the environment. One way to do this is to use a suitable base landcover palatte as suggested in the article, combined with relief shading and textures to represent the environment to the fullest extent.
The use of hypsometric tints is debated. Personally I think that every map feature is fit for a purpose and hypsometric tint are no different. It is deciding what you want your map to show, and how you are going to combine these features into a single map to make it multi-purpose.
Hypsometric tints are useful for showing elevation levels (no surprise there). So for showing points above a certain altitude where the air may be getting thinner, or comparing which of these two mountains is higher, or by gauging if this is just a small hill or Mount Everest.
Contours are good for showing steepness (how close the lines are together) and identifying the gradient of a paths by if the path runs parallel or orthogonal to the contour line.
Shaded relief is great for getting a general sense of where the hills are and their form.
Physical coloured landcover combined with textures or representative bump maps give you a feel for the environment.
The next task is building up a physical map which gets applied to OSM data (possible in tandum with other free sources) which uses natural colours, shaded relief, textures and bump maps. The only question is whether to build something like the Stamen watercolour maps but with natural colours and textures, or instead work towards a natural style for osm2world (or any other osm to 3D world program).
A free and open source SRTM shaded relief and contour map
I’ve recently pushed out a new srtm3-stylesheets repository which contains shell scripts for working with NASA SRTM DEM data, gdaldem based stylesheets for shaded relief maps, Mapnik stylesheets for contours and a TileStache configuration for sandwich those styles together into a single map.
This was spurred on by the fact that I simply needed a map which showed hills. I’ve used Andy Allan’s OpenCycleMap in the past which has color relief and contours. Unfortunately it is closed source.
So I put my head down and hacked together repeatable scripts to get the source data up and running and some basic stylesheets to produce a usable and pleasant looking map. All released as free and open source software under the CC0 license.
I want to avoid adding things like streets etc, such maps could be built as separated layers based upon this style and sandwiched together, for example, with the TileStache sandwich provider.
I hope to build upon the lessons learnt here to produce a map like the Stamen Terrain map, except with the source code released under a free and open source license. Perhaps just with hill and slope shading applied to landuse with other map features placed on top.
I’ve rendered NSW (only server resource prevent worldwide!) as a slippy map here.
Making a Shaded Relief Map from the Geoscience Australia Australian Bathymetry and Topography Grid
I’ve just released some scripts and stylesheets for the Geoscience Australia Australian Bathymetry and Topography Grid data.
Here are some samples of the final outputs. (also available as a layer on my demo slippy map)
These images based on data © Commonwealth of Australia (Geoscience Australia) 2009, which was released under the Creative Commons Attribution 2.5 Australia Licence, http://creativecommons.org/licenses/by/2.5/au/.
The code I used to produce these images is CC0, so as far as I am concerned you are free to recreate them under whichever license is compatible with the upstream data. If you just wish to use these images, I license the ones you see here (so only if you grab them from here, rather than re-rendering them) under the Creative Commons Attribution 3.0 Australia license, http://creativecommons.org/licenses/by/3.0/au/
Unlocking the Landsat Archive project
The impacts of the Geoscience Australia switch to Creative Commons licensing is starting to take more shape. With much better quality licensing statements and notices appearing with new works they publish.
The licensing of the AGRI has allowed members of the public to on their own autonomy make and release derived works such as agri.openstreetmap.org, or the OSM edits which use it. GA must have problems hosting the huge amount of data themselves (because you can only obtain it via DVD), no problem the CC licensing has allowed the public to do a better job with hosting it online.
I’ve done a similar thing for the The National Dynamic Land Cover Dataset.
In other news I’ve just learnt about the Unlocking the Landsat Archive project. This is interesting because “Within this project, Geoscience Australia will transfer it’s archive of Landsat data to the NCI and make it freely available under a creative commons licence.”
Aiming to be completed by June 2013, lets hope they make both the raw data and standard derived products available. All under CC-BY or CC0, and all through a documented API which developers can use to access the data.
Finally I’ve found http://earthengine.google.org/. I’m really impressed. It is really easy to use, you just select the satellite and your date and up on your web map you get the collected imagery overlaid.
All this raster work has rekindled my interest in remote sensing.
ASGS 2011 Mesh Block Category (landuse) as a Pie Chart and a Map
Looking at the ASGS 2011 mesh blocks geographically, most of the landmass of Australia is classified as Agriculture (I am aware that the mesh block category which is used here isn’t a definitive land use, purely an indicator of the main planned land use of the mesh block…). Only a few coastal areas, the hills between Vic and NSW at the South East (I should really know the proper name… better have the name added to (F)OSM!) and West Tasmania are Parklands.
Using some scripts I wrote, I present a graph of the mesh block total area by category.
Letter to the ABS Re Census DVD License
Recent code projects
Since taking a break from OSM/FOSM[1] I’ve found myself with more time to work on other projects. You can see this by my jump in recent activity on github[2].
I’m pleased with my code to load the ABS ASGS into PostgreSQL/PostGIS[3] and that will prove a critical component of any further work I do with ABS data in the future. At the moment I’m writing scripts to load ABS 2011 Census data into PostgreSQL[4] (but I haven’t pushed it yet). I’m using the sample datapacks provided by the ABS which means I should have it done by June 21 when the actual data is release. So with very minimal work and time I should have the real data loaded not long after it’s release.
I think http://www.hackdays.com/knowwhereyoulive/postcodes/view/2000 is an excellent site, yet the creators say “The biggest hurdle we faced was the format of the data available.”[5]. I hope that my asgs2pgsql and abs2pgsql (census loader) tools will help future developers built great front-ends, and not have to worry about getting access to the data in a nice format.
I’ve been playing around with the GeoJSON tile capabilities of TileStache (with Polymaps as the client) and look forward to using this in building a great front-end like the knowwhereyoulive site. I’ve put that on hold while I focus on the data.
I’ve also ordered the first DataPack DVD[6] from the ABS. I’ve been assured by the ABS that the DVD contents are licensed CC-BY and the license will be present on the DVD. Assuming all this holds true I should be able to release the DVDs as torrents.
I plan to attend the Sydney http://www.govhack.org/, it is just a pity that the event happens before the 2011 Census data is released. I’m not convinced that these hack days are for me…(I prefer to take as much time as needed to get quality, rather than rushing things out the door; also borrowing a laptop and setting up a live bootable debian USB with my environment and wifi drivers takes more time than the actual event! ) but at least they get people talking about open data.
I’ve also produced some nice rainfall intensity interactive graphs from the BOM radar images[7]. I wish I could show a demo of this, but the free and open license to the radar image isn’t present.
I also have some other data loaders [8], [9] which I plan to create nice web front ends for, but I’ll only get to that once I get my 2011 Census loader in better shape.
Finally I took a sidestep to clean up my NSW Parish Map scraper[10]. The code is much cleaner now compared to the mess it was before. I’m still short of a parish map georeferenced base map mosaic, but the unclear licensing of the maps puts this down the bottom of my priority list.
[1] https://andrewharvey4.wordpress.com/2012/04/04/the-sand-storm/
[2] https://github.com/andrewharvey/
[3] https://github.com/andrewharvey/asgs2pgsql
[4] https://github.com/andrewharvey/abs2pgsql
[5] http://www.hackdays.com/knowwhereyoulive/about/
[6] http://www.abs.gov.au/websitedbs/censushome.nsf/home/datapacksdetails?opendocument&navpos=250
[7] https://github.com/andrewharvey/bom-rainfall-radar-intensity-stats
[8] https://github.com/andrewharvey/aec2pgsql
[9] https://github.com/andrewharvey/ausgrid-data2pgsql
[10] https://github.com/andrewharvey/nsw-parishmaps-scraper
The Australian Bureau of Statistics… open data but closed documentation for that data?
Update: The HTML equivalent of the PDF is available at http://www.abs.gov.au/ausstats/abs@.nsf/Latestproducts/2901.0Main%20Features12011. Unlike the PDF, the HTML version is licensed under the Creative Commons Attribution 2.5 Australia license (and confirmed via email). I prefer the HTML version, I just hadn’t noticed it earlier. Thanks to the Client Support Officer from the ABS who got back to me with this information.
Just a day after I posted my ASGS to PostgreSQL loader and plans to integrate with the ABS data, I’ve already hit resistance.
Once piece of documentation the ABS released for the 2011 Census data at http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/2901.0Main%20Features802011/$FILE/2011%20Census%20Dictionary%2027102011.pdf (ABS Catalogue No. 2901.0) contains the full copyright notice without any reference to it being licensed under a free and open license such as the Creative Commons Attribution license. This notice is an “unless otherwise noted exception” to the CC license on the ABS website.
I made an enquiry to the ABS about this and was told “Written permission is required from the ABS if you wish reproduce this publication or extracts from it.”
I’m disappointed that the ABS refuse to release their documentation for the 2011 Census data under a free license such as a Creative Commons license. It doesn’t really make sense to me why they would release the data under an open license, but not release the documentation under the same or equivalent license. This only hinders the availability of the documentation and would lead to data consumers needing to make guesses or oversights about data and as such would be more likely to misinterpret it.
Google Reader Shared Items
- An error has occurred; the feed is probably down. Try again later.
License
The content on this blog by Andrew Harvey, unless otherwise noted is licensed under a Creative Commons Attribution-Share Alike 3.0 Australia License.
Andrew Harvey's Blog 
