Blog moved to new host

All my WordPress blogs have moved to my own hosting site. Visit the site home at http://converser.nz

NZ Rail Maps Project Development Report [2020K]: Web Development Trial

The above is the home page you will see when you put the address

http://live.nzrailmaps.nz

into your web browser. It is starting as a default WordPress site and as Project spends time customising it, will change in appearance to resemble this blog.

As of today, Project has signed with GoDaddy NZ for 1 year for a special discount price of $20.56. This gives a Linux hosted site with plenty of space for hosting both this blog and web tile maps. The site will thus have two parts:

• Documentation, using the included WordPress blog. This includes all existing blog posts on nzrailmaps.wordpress.com that will be migrated to the new site.
• Web tile maps: a website to provide online viewing of NZ Rail Maps content. This is a new development of NZ Rail Maps Project that has been discussed in recent posts.

The site URL shown above is a temporary URL used for development purposes to allow the existing website at http://www.nzrailmaps.nz (hosted on SmugMug) to continue operating as it is at present alongside this website. Project will allow SmugMug site to stay up as long as necessary (not more than 3 months) enabling the new site to be developed alongside it. However, further development of the existing site will essentially be frozen from this point going forward.

The new site will become the default web site for http://www.nzrailmaps.nz as soon as it is developed enough to replace the existing SmugMug site. Project does not know when this will take place since it is necessary to replace all of the existing content on SmugMug. It is possible as an interim step the existing maps map be migrated back to Google photo albums whilst the web tile maps are developed so that SmugMug site can be shut down soon.

Project has a test version of web tile maps for Northland ready to go onto the demo site within the next few weeks for further testing. In fact this could be up within days. This is where Volume 1 content will appear going forward. The site will go through various stages of development as there is a lot of work to get something that looks like the type of content currently hosted with SmugMug, especially with aerial photos. It is likely the site will incorporate historical mosaic photos as part of the included web tile maps, but the best way of serving current aerial photos is probably by bringing them in live from the Linz Basemaps site rather than by including them in the web tiles hosted on the nzrailmaps site.

Stay tuned for further updates on development of the new web site for NZ Rail Maps Project. This will be staged as a lot of work is needed and probably the focus over the rest of this year is going to prioritise migrating some form of the existing content from SmugMug to enable it to be shut down soon. Some of the SmugMug functionality may be lost in the short term and reinstated later. Project does not undertake to give a particular timeframe for development. There are many steps of development that will take a lot of time to implement alongside the task of progressing with Volume 1 itself. There is still a goal of completing Volume 1 which has been shifted to the end of 2020 to allow the web site to be developed to host it when it is ready for production. The alternate to that is that Volume 1 might actually be ready a lot sooner but only be hosted in a more basic form.

Printable content is going to be the only truly offline form of content for the site since it will not be straightforward to download static images for the site in the way that can now be done with the photo album format on http://www.nzrailmaps.nz . Project will make a decision soon about format of these whether Kindle, printed or both.

NZ Rail Maps Project Development Report [2020J]: New Production Options

In Project’s last PDR, [2020I], published in August, the development of the nzrailmaps website that had been started at http://www.nzrailmaps.nz was discussed. The site development discussed there is for a static website where the maps are manually generated from the GIS and then uploaded as images to a photo gallery type of site. Project chose SmugMug to host this because of previous experience using the Flickr platform to host photo galleries, which has been taken over by SmugMug. Currently a SmugMug commercial hosting package is being used on a monthly basis, equating to around NZ$19 each month, plus domain hosting of around $30 a year. The site blog appears separately as a free WordPress site.

In recent reflection of some issues and challenges associated with SmugMug, Project undertook further investigation with alternative considerations, which has been generally rewarding. Firstly in hosting cost, an example being GoDaddy NZ coming down to $11/mth. Secondly in what can be hosted. The option exists to either continue with a photo gallery format or tile based maps that can be zoomed and dragged just like Google Maps, which is pretty well supported using the WMTS or XYZ standards. Thirdly, in charging for printed volumes.

There is a lot more work to do on looking at all of these options, but so far, testing with XYZ tiling is proving very rewarding. If this option was pursued, a new website would be developed to replace the current one, which would remain as is. It would then be a question of what ended up being hosted on a web site. Currently, the online maps are being generated using aerial photo backgrounds. It does not make sense to replicate the Linz aerial photo layers that can be dynamically added into a map. Hence, Project would seek through a web site to make it possible for the maps to be overlaid directly onto the Linz Basemap aerial layer. It would also very likely look to generate tiles for the historical aerial mosaic layers and make them able to be displayed as backgrounds as well.

Project would also aim to create printed volumes for those who prefer an offline format, an important consideration with a change in online format from static images to tiles, since it is no longer feasible to download a set of static images that can be browsed on a device offline. There would be a charge for hardcopies.

There are many considerations, but a new web server could be running early next year (with a realistic timeframe for implementation) and host Volume 1, which would not be produced in the current format. In other words, apart from the samples of Volume 1 which have been produced to date, static images are not going to be made available. Project would aim to migrate all existing volumes (1, 5 and 6) and cut over to the new website at that time. Existing content would be shifted to Google Photos until such time as the web site option is fully implemented.

There is a lot of info there. For now, Project will continue to work on completion of Volume 1 as before, but Volume 1 will not produce a static map set the same as existing Volumes 5 and 6 which are currently hosted on the SmugMug site. It is expected that Volume 1 will become the first to move to tile based on a new web server regardless of the completion timeframe. There are a lot of steps to be completed and these will be part of Volume 1 publication once the actual mapping is completed in the GIS. Hence, Project does not now undertake a specific timeframe for publication of Volume 1 before the end of 2020.

What is relevant is that development levels (Basic / Intermediate / Comprehensive) will still be important as the initial state of web based development will be the Basic level for all volumes and assuming the new web hosting format is implemented, all volumes will be implemented there as the most important priority for the maps. As previously stated, Volume 1 will be completed at its current level, however the extent of publication of historical aerial map mosaics depends on the ability to implement them in the proposed new web tile formats. This is yet to be determined.

Project will continue development of Volume 1 as existing, with regular updates on the the groups.io development list, and progress summaries as as present; these are likely to include further information on the topics raised in this post as the ideas are explored further and tested. Project looks forward to ongoing support from the existing community, and continues to explore setting up a formal entity to oversee and finance development.

North Auckland Line [0W]: Weekly Progress Report – Week 39, 2020

Project recently has completed some mapping of Okaihau / Rangiahua sections, Dargaville / Kaihu sections and North Auckland Line main route.

However due to other life priorities work has slowed during the past week and Project does not expect the present very slow progress to show much increase during the current week. Work has started on basemaps of the Dargaville Branch which is very hard work because of very poor quality aerial photography of the area. The Northland Regional Council does not appear to greatly value good quality aerial photography and now has the fuzziest in NZ despite it being flown only about five years ago. As the historical aerials of this area have been completed, they are ready to be introduced into the GIS and copied.

Project is looking at hard copy publication formats and costs and it cannot be ruled out that small run publishing might result in hard copies of volumes being available for sale in future. Project believes cost is about the same whether using Amazon CreateSpace per-copy printing or local printers in runs of 10. However, Project would have to be set up as a formal entity and this has been considered for some time and remains under consideration.

Project’s overall schedule remains important and with the expectation being changed lately that Project will be wound up or completed within two or three years, into more of a long term activity, it is likely Project will shift focus going forward into producing Basic level online maps in every volume as the priority as well as hard copies (which are also Basic level), in order to allow more time to complete the much more intensive intermediate / comprehensive levels. The drawn out process to complete Volume 1 has made this reconsideration necessary.

Earlier in the year, Project made good progress with Volumes 5 and 6, but as the year has worn on, the advent of the global pandemic has had sufficient impact to slow the schedule on multiple occasions. This has nothing to do with any economic situation. Project is receiving more funding than ever before and recent expansion of community support has given assurance that other sources of funding may be tapped into in future. What essentially the Covid-19 impact has caused this year is to personal schedules and time available to complete Project’s work. There have been more than a few weeks lately when only small amounts of work were completed and expectations were not met. This is unlikely to change in the forseeable future and the only reasonable outcome going forward is to revert to the earlier idea of staged completion through the use of variable levels of development for each volume. Project still desires to get content out into the community and increase knowledge of the work being done and this can really only be achievable by completing every volume as quickly as possible, which most likely means at Basic Level only except for those volumes already completed at Intermediate / Comprehensive Level.

Once Project as a whole is back to a higher rate of production, maybe later this week but it could take longer, Project will continue to review options for greater community support, scheduling and stages of development for the entire work. The current level of development of Volume 1 will continue as the majority of historical aerial photography for the planned intermediate / comprehensive levels has now been completed. It is however uncertain when this volume will actually be finished. It could take until the end of October or it could take until the end of the year. At this time, Project has no desire to be bound to an absolute deadline at what is traditionally a busier time of year for most people. Furthermore, Project will probably by the end of October only have completed research and map design. Actually producing the thousands of individual maps in Volume 1 could take another month as it often shows up more work that is needed.

North Auckland Line [0V]: Weekly Progress Report – Week 38, 2020

Apologies for failing to produce these posts for the previous two weeks. Since the last update the following has been completed:

• Maps drawn of Portland Station, the site of the Wilson Portland Cement Company, now the only cement manufacturer in NZ, which once had its own private railway network and numerous locomotives it owned to work on the tracks over almost a century.
• Complete the next section of mainline corridor from Waotira to Portland. This means it is almost up to Whangarei, the next station further north.
• Taken a side trip to Whanganui to draw a revised set of maps of the old Taupo Quay station and the Castlecliff Branch having discovered a NZR corridor survey from Aramoho to Castlecliff.
• Completed a set of mosaics for stations on the Dargaville and Kaihu branches. Full maps of these routes will be drawn in the coming week.
• Created a set of historical mosaics for the Okaihau extension. Okaihau is sometimes touted as NZ’s most northerly railway station – however Opua is actually slightly further north. The extension to Rangiahua via Puketi and Okoro was constructed in 1926 following completion of the Otiria-Okaihau railway and was intended to reach Kaitaia. It was never formally handed over to NZR and was closed in the late 1930s with the rails lifted around 1940 to be used elsewhere. Having found 1942 and 1953 old coverage, 1961 and 1986 also being included for reference, the latter being highway surveys, it is expected to produce really good maps of this section. The line from Otiria to Okaihau was closed in 1987 and the track lifted, the corridor is now being used as a cycleway.
• Started editing the existing set of mosaics for Otiria-Kaitoke and Otiria-Opua. It needs to have Opua and Kaitoke added back in, as the layers have either been lost from the file due to corruption, or were in a separate file that doesn’t exist any more.

With the progress over the past week the mosaics for the rest of the North Auckland rail corridor are nearly all complete or should be completed this week enabling mapping to push ahead. The remaining areas to be mapped are:

• Main line from Portland to Otiria, 78 km
• Branch from Waiotira to Dargaville, 47 km
• Branch from Dargaville to Donnellys Crossing, 34 km
• Branch from Otiria to Opua, 17 km
• Branch from Otiria to Okaihau, 39 km (plus unknown extension length)

Therefore maps required to cover about 220 km. This is quite a lot considering the main line corridor has a total length of 281 km and 203 km of that has already been completed. It means that the total length of railway network in the NAL corridor and Volume 1 of these maps is about 420 km, so actual mapping is only a little past half way. However, mapping is the quick part of the process – the historical mosaic work is the slow time-consuming part. The previous deadline for completing the project was the end of September. At the moment it is unclear how much progress towards that can be made, but a month is a more realistic expectation.

North Auckland Line [0U]: Weekly Progress Report – Week 35, 2020

Well last week we took things a bit more slowly and so we got only a little bit of maps work done. Whilst we are still just as busy with other stuff this week, better use of our time means a lot more mapping will be completed over the coming days. Currently, mosaics are being put together for Dargaville/Kaihu branches and we are also creating them for the section of the main line from Oakleigh through to Waro (specific stations, not continuous). We have more or less completed the additional maps for Auckland and this will produce the first post this week. So expect to see more maps completed in the coming week and posts to groups.io coming up.

North Auckland Line [0T]: Weekly Progress Report – Week 34, 2020

So the maps are now completed to Waiotira, which is 179 km from Westfield and at the junction of the Dargaville Branch from the NAL. In the last three weeks we drew the maps from Helensville, a track mileage of around 115 km which is quite a good achievement, but not as fast as we have done in earlier periods of this year due to time use constraints.

Whilst we expect other demands on our time are unlikely to lessen for the rest of this year, we remain committed to completing Volume 1 at the earliest possible opportunity, which due to our new content policy, is now expected to occur around the end of September. We have also done some updates to maps covering some of the NAL corridor within Auckland City.

Having completed and published sample maps all the way to Waiotira, we are now taking a break for three days to address other current priorities in our schedule so the “railhead” will not be advanced beyond Waiotira during this period. The work which we are expecting to take the rest of the week to complete will include at least some of the following:

• Aerial mosaics for the Dargaville and Kaihu Branches
• 1940 and 1955 aerial mosaics for the rest of the section from Mt Albert to Swanson in Auckland (so far only New Lynn to Sunnyvale has been completed).
• In the section just completed from Maungaturoto to Waotira, we intend to add mosaics for Huarau, Tunnel 11 and Waikiekie. The two mentioned stations had industrial sidings and so will be mosaiced in accordance with our policy for the creation of historical maps. Tunnel 11 is the Golden Stairs Tunnel which has an abandoned tunnel near the current one, which is a left over relic from construction days.

So during the next three days there definitely will not be any new sections mapped. If any maps are published during this time, they will be updates of areas already mapped. Also the next section to be mapped is the two branch lines mentioned above and mosaics are needed for them before proceeding.

Based on our schedule so far therefore we would expect by the end of August we will have completed the two branch lines, and possibly the main line as far as Waro. However this depends on how much work we need to do on the aerial mosaics for the Whangarei area.

The project web site, http://www.nzrailmaps.nz/ has been updated to contain a gallery for each station, and the folders for each station are accordingly linked from the home page. During the development phase, sample maps created for each station are being placed into these folders. When the final maps are published, the maps for each station will be collected from the map volumes into these folders. This enables a station to be quickly located by name, whilst linking to the maps produced for the volume that the station is within.

NZ Rail Maps Project Development Report [2020I]: Web Site Development – August 2020

As it is generally known, we have a web site at http://www.nzrailmaps.nz. The website is in its early stages of development, and we have still to determine various parameters for it. We are, however, expecting to carry out the following content development options:

• Add volume galleries of the full set of maps for all of the 12 volumes that cover New Zealand.
• Create station galleries to find station information easily. These will include collected maps from the volume galleries, and it might include other documents such as S&I diagrams, source aerial photos or geotagged mosaic tiles. * Currently due to size, aerial photos are likely to be excluded, and only crops of photos for stations that are not mosaiced are likely to be placed in station galleries.

We expect the current homepage to be significantly improved, and it will be linked into all volume and station galleries. At the moment it has only been linked into the volume galleries that have been completed so far (Volume 5 and 6 at the time of writing).

The intention for content placement is to place the original content of the maps project for the most part and we aren’t currently planning to place third party content in any quantity. It takes a lot of work to collect or create content, and it is less of a priority when it isn’t directly related to the project. We may place some third party content into another site such as Google Photos, which we will consider as the project goes on, but it may well be that the only content on the site is content that has our copyright banner on it, to avoid any appearance of problems over copyright of third party content. The volume of disk space usage for each completed volume is significant; to date the two volumes published have used around 5 GB each and that implies we could easily end up using 50-60 GB for the maps for all of the country.

Hence the development of the web site will follow this pattern:

• As a volume is developed, a station gallery will be created for each station and sample maps placed there.
• When the volume is published, the full set of maps for the volume will be placed in the volume galleries
• The published maps for each individual station will then be collected from the volume galleries, and sample maps removed.

NZ Rail Maps Project Development Report [2020H]: Development Schedule Latest Revision

Over time we have looked into different map development schedules. We first attempted about two years ago to prioritise producing a Basic version of all 12 volumes as a high priority to be completed within months. That however has been re-examined a few times. We have looked at different compromises between getting the maps finished as quickly as possible, and spending as much effort as we can on getting everything into them that we think should be documented, especially old historical content.

Our most recent thoughts have been on a multi stage development of each volume, with a Basic stage completed as quickly as possible, and a more in depth revision completed later. However this has never been easy to achieve in practice. We have, accordingly, pretty much settled down to our current schedule proposal as below.

We now propose that only one step be made on completing all volumes, which is to as full a level as we consider reasonably practicable. This means each volume will take months to complete rather than weeks. There are various reasons why we have examined different schedule proposals in the past. However, we now feel that because of the significant amount of resources which have to be duplicated across revisions, and given the scarcity of our resources such as time in general, we are going to go with one stage of development for all volumes.

This isn’t intended in any way to push the schedule for Volume 1 out, it just means we are not attempting to cut out content. We do have a definite view of how much more work is needed to complete Volume 1, and we are reasonably pushy about getting it finished by the new timeframe of close to the end of September. But there won’t be another revision of it to a more in-depth level planned in the future. It will be completed to as full a standard as we feel is reasonably practicable at this time, and we aren’t planning to come back in the future for another full revision unless significant new content sources are located.

Because of this decision, Volume 1 is currently having new content added within Auckland City, as well as up in Northland. This has pushed the completion out somewhat and as we have suggested, the end of September is more realistic for the present. We will keep pushing out daily updates to our email list for each station as maps of these areas are completed.

North Auckland Line [0S]: Weekly Progress Report – Week 33, 2020

Since our last report we have advanced the base maps to Kaiwaka at about 140 km from Auckland, which is practically half the entire distance from Westfield to Otiria. This represents good progress under our current rate of resourcing for map production. At the time of our last post we had just completed maps up to Helensville so progress has been steady but perhaps not as spectacular as at other times during this current year. We remain focused on our key objectives of completing all 12 volumes of maps within a three year period from 2020 to 2022, but the two stage approach we have considered using is probably not going to be followed. We respect that there is a considerable demand for these maps to be produced and available to the community at large, but feel that producing at two levels over two different periods for each map, is probably not what we are going to be motivated to do, and that finishing each volume in one go is likely to be the key strategy employed over this period.

To that end, we are continuing to produce new historical mosaics in Volume 1 and in the recent period have been working on a set covering the Dargaville and Kaihu branches, and additional stations on the main line, including Te Hana, Kaipara Flats and Kaiwaka. Pre-production samples of these will continue to be published in Daily Diary entries on the group’s email list. We have not yet produced any significant quantity of final maps for the NAL rail corridor because issuing the sample maps meets the needs for publicity at developmental level and allows us to make corrections to the maps as the samples are created, prior to producing the final maps for official publication on our web site.

The next samples to appear on our email list will be Kaipara Flats and Kaiwaka. The next station after Kaiwaka will be Maungaturoto, for which mosaics have already been generated, probably a day after the former as it is only 10 km further up the line. After that, it will take another day or two to reach Waiotira at around 180 km and then we can hopefully spend a little time but not too much mapping the stations for which we have mosaics on the two branchlines west of Waiotira. Then back on the mainline, we head up to Oakleigh (the Marsden Point junction), Portland, Whangarei, Kamo and Kauri. Although Whangarei is a major metropolis in the north, its railway infrastructure is relatively confined into the city yard and the wharves and mapping these should not take too long.

As stated on the email list, we will add maps of Hikurangi, just north of Kauri, which had an industrial siding and remained open until quite recently, and Waro, where there were coal mines with sidings, to the maps. After that there will be nearly 50 km of route to cover without pause to Otiria. From there the completion of Volume 1 will be achieved after taking in the two branches, in the east to Opua and in the west to OKaihau. We have already mapped much of the historical features of these lines and are not planning to look into any more unless new sidings or major infrastructural facilities are located, which is unlikely to take place.

Taking all the above into consideration we are less confident about forecasting a completion date, whilst at the same time remaining committed to completing the volume. We estimate at this stage that by the end of August we should have reached Hikurangi after completing the rest of the main line corridor north of Kaiwaka including Whangarei and the two branch lines west of Waiotira. A reasonable guess is that it would take up to another two weeks to complete the rest of the main line and the two additional branches. It is then necessary to schedule additional time to produce the final maps for publication, so a completion date in September now looks more likely for Volume 1.