"I am my choices. I cannot not choose. If I do not choose, that is still a choice. If faced with inevitable circumstances, we still choose how we are in those circumstances." – While this quote is widely attributed (as an English translation) to Jean-Paul Sartre, I can't seem to validate it in any kind of original French-language text. Certainly he said something similar, though.
[daily log: walking, 2km]
Kiamon never once thought on her fate
Grimly she battled to push down her hate
Hoping perhaps to at last find her goal
Kiamon willingly gave up her soul.
…Recently I tried something new. As some of you know, I have a rather wide set of “novels in progress,” none of which actually progress, much. I’m bad at these wider, longer-scale projects. So I decided to take this slightly more successful short poem-a-day concept and “hijack” it for the novel thing. I have been writing little “character-building” quatrains, where I try to encapsulate some aspect of a story’s character. This is one of those. In general, don’t be surprised to see the names of fictional beings begin to populate some of my poems.
[daily log: walking, 1km]
The Faëry Chasm
No fiction was it of the antique age:
A sky-blue stone, within this sunless cleft,
Is of the very footmarks unbereft
Which tiny Elves impressed; – on that smooth stage
Dancing with all their brilliant equipage
In secret revels – haply after theft
Of some sweet Babe – Flower stolen, and coarse Weed left
For the distracted Mother to assuage
Her grief with, as she might! – But, where, oh! where
Is traceable a vestige of the notes
That ruled those dances wild in character? –
Deep underground? Or in the upper air,
On the shrill wind of midnight? or where floats
O'er twilight fields the autumnal gossamer?
– William Wordsworth (English poet, 1770-1850)
[daily log: walking, 7km]
I spilled some water
there on my floor. Then I stepped
in it. What is that?
Nerds from many countries bring us the future, one step at a time.
[daily log: walking, 7km]
Thirst.
Some nights…
I wake up
from restless dreams,
my mouth dry, broken.
So I get some water,
and pace my apartment's floor,
digesting the dissolving webs
of grimly inchoate chimeras.
We live in a weird era. Entropy has become a kind of commodity in and of itself.
In all this messing around with my new server… with trying out new things and tinkering with it all… well, of course I have to educate myself a bit about server security. It's a big, bad world out there, and if I'm going to be running a server that's publicly visible on the internet (offering up webpages, etc.) the little machine will be lonely and vulnerable, and I have to think about how to protect it from bots and blackhats.
In the field of network security, one thing that comes up is that you have to have some fundamental understanding of the types of cryptography used these days to secure systems. There's a whole infrastructure around generating "secure" public and private keys that computers hoard and exchange with one another to authenticate themselves. I really DON'T understand this, but I wade through the documentation as I e.g. try to set up a certificate authority on my server, because some of the things I'm running there apparently require it. I run the commands they tell me to, and hopefully my little server is sorta secure. But who knows.
I was fascinated to learn, however, about a thing that is used in crypto key generation on computers: system "entropy."
On one site I was looking at, there was a discussion about the fact that virtual machines (the sorts you rent from big companies to run cheap little servers, as I have done) have extremely low "available entropy" while your typical crummy desktop has very high "available entropy" – therefore when I generate my keys, I should do so on my desktop, not my server – I can upload the generated keys to my server later.
I think it's kind of a funny concept. The mass-produced, cookie-cutter, high quality, reliable servers found on the giant server farms are lacking in a certain commodity that they desperately need for their security: entropy. So the admins have to go out to their desktops to get the entropy they need. I sit here and I listen to my cruddy, 7 year old Jooyontech Korean PC-clone desktop, with its perpetually failing CPU fan groaning intermittently and the weird system noises filtering though the sound channel onto my speakers, and I can rest assured that that's all part and parcel of having lots and lots of good, tasty entropy that I can feed to my server in the form of so many sweet, generated security keys.
One site I was reading said that typical desktop entropy should be around 2000 (in whatever units entropy is measured with…).
Out of curiosity, I plugged in the Linux command that would tell me my system's entropy. I got 3770. Wow! I'm rich! … in entropy, anyway.
Meanwhile, my server, a virtual machine in some well-air-conditioned server farm facility across the Pacific in California, manages only 325 units of entropy. So sad. The chaos-poor, withered fruits of conformity.
[daily log: walking, 7km]
Sometimes the day starts
with a sense of frustration
but ends feeling fine.
My 5th grade student John recently started at Karma. He has never studied English before, so in the context of Korean English education, he's a rather "late starter." It's hard to place him at a place like Karma, because classes at his grade level don't really have "beginner status". Still, I was very proud when he produced the following comic during an exercise in class. He carefully asked (in Korean) how to say each thing in English, and I wrote the translations on the board. Then he copied the letters (he hasn't even mastered the alphabet) into his comic. I thought his characters (an imitation of my alligator character, along with some lion character he created), while quite rudimentary, were cute, too. He might have some artistic talent, anyway.
[daily log: walking, 7km]
The night is too thick:
Highrises' lights vague and dim;
Air stiflingly chill.
[The following is a direct cross-post from my other blog – just so you don’t think I’m doing nothing with my free time, these days.]
[Update 20180923: continues from here]
I started discussing the coastline shapefile problem in first post.
Early on, I found the tool called QGIS Browser and installed it on my desktop. I used this to examine the shapefiles I was creating.
The first step was to look at the “real Earth” OpenStreetMap-provided shapefiles I was trying to emulate – the two mentioned in my previous post:
/openstreetmap-carto/data/land-polygons-split-3857/land_polygons.shp
and
/openstreetmap-carto/data/simplified-land-polygons-complete-3857/simplified_land_polygons.shp
Here are screenshots of each one.
First, land_polygons.shp
And here is simplified_land_polygons.shp
The structure is pretty straightforward, but – how do I make these? Where do they come from? – aside from the non-useful explanation found most places, which is that “OpenStreetMap generates them from the data”.
The way that the shapefiles are generated for OpenStreetMap are not well documented. But after looking around, I found a tool on github (a software code-sharing site) developed by one of the OpenStreetMap gods, Jochen Topf. It is called osmcoastline, which seemed to be the right way to proceed. I imagined (though I don’t actually know this) that this is what’s being used on the OpenStreetMap website to generate these shapefiles.
The first thing I had to do was get the osmcoastline tool working, which was not trivial, because apparently a lot of its components and prerequisites are not in their most up-to-date or compatible versions in the Ubuntu default repositories.
So for many of the most important parts, I needed to download each chunk of source code and compile them, one by one. Here is the detailed breakdown (in case someone is trying to find how to do this).
I followed the instructions on the github site (https://github.com/osmcode/osmcoastline), but had to compile my own version of things more than that site implied was necessary. Note that there are other prerequisites not listed below here, like git, which can be gotten via standard repositories, e.g. using apt-get on Ubuntu. What follows took about a day to figure out, with many false starts and incompatible installs, uninstalls, re-installs, as I figured out which things needed up-to-date versions and which could use the repository versions.
I have a directory called src on my user home directory on my server. So first I went there to do all this work.
cd ~/src
I added these utilities:
sudo apt-get install libboost-program-options-dev libboost-dev libprotobuf-dev protobuf-compiler libosmpbf-dev zlib1g-dev libexpat1-dev cmake libutfcpp-dev zlib1g-dev libgdal1-dev libgeos-dev sqlite3 pandoc
I got the most up-to-date version of libosmium (which did not require compile because it’s just a collections of headers):
git clone https://github.com/osmcode/libosmium.git
Then I had to install protozero (and the repository version seemed incompatible, so I had to go back, uninstall, and compile my own, like this):
Git the files…
git clone https://github.com/mapbox/protozero.git
Then compile it…
cd protozero
mkdir build
cd build
cmake ..
make
ctest
sudo make install
I had to do the same for the osmium toolset:
Git the files…
git clone https://github.com/osmcode/osmium-tool.git
Then compile it…
cd osmium-tool
mkdir build
cd build
cmake ..
make
That takes care of the prerequisites. Installing in the tool itself is the same process, though:
Git the files…
git clone https://github.com/osmcode/osmcoastline.git
Then compile it…
cd osmcoastline
mkdir build
cd build
cmake ..
make
I had to test the osmcoastline tool:
./runtest.sh
So now I had to try it out. Bear in mind that each command line below took several hours (even days!) of trial and error before I figured out what I was doing. So what you see looks simple, but it took me a long time to figure out. In each case, after making the shapefile, I would copy it over to my desktop and look at it, using the QGIS browser tool. This helped me get an in intuitive, visual feel of what it was I was creating, and helped me understand the processes better. I’ll put in screenshots of the resulting QGIS Browser shapefile preview.
To start out, I decided to use the OGF (OpenGeofiction) planet file. This was because the shapefiles were clearly being successfully generated on the site, but I didn’t have access to them – so it seemed the right level of challenge to try to replicate the process. It took me a few days to figure it out. Here’s what I found.
Just running the osmcoastline tool in what you might call “regular” mode (but with the right projection!) got me a set of files that looked right. Here’s the command line invocation I used:
YOUR-PATH/src/osmcoastline/build/src/osmcoastline --verbose --srs=3857 --overwrite --output-lines --output-polygons=both --output-rings --output-database "YOUR-PATH/data/ogf-coastlines-split.db" "YOUR-PATH/data/ogf-planet.osm.pbf"
Then you turn the mini self-contained database file into a shapefile set using a utility called ogr2ogr (I guess part of osmium?):
ogr2ogr -f "ESRI Shapefile" land_polygons.shp ogf-coastlines-split.db land_polygons
This gives a set of four files
land_polygons.dbf
land_polygons.prj
land_polygons.shp
land_polygons.shx
Here is a view of the .shp file in the QGIS Browser. Looks good.
I copied these files into the /openstreetmap-carto/data/land-polygons-split-3857/ directory, and I tried to run renderd. This alone doesn’t show the expected “ghost” of the OGF continenents, though. Clearly the simplified_land_polygons.shp are also needed.
So now I experimented, and finally got something “simplified” by running the following command line invocation (note setting of –max-points=0, which apparently prevents the fractal-like subdivision of complex shapes – technically this is not really “simplified” but the end result seemed to satisfy the osm-carto requirements):
YOUR-PATH/src/osmcoastline/build/src/osmcoastline --verbose --srs=3857 --overwrite --output-lines --output-rings --max-points=0 --output-database "YOUR-PATH/data/ogf-coastlines-unsplit.db" "YOUR-PATH/data/ogf-planet.osm.pbf"
Again, make the database file into shapefiles:
ogr2ogr -f "ESRI Shapefile" simplified_land_polygons.shp ogf-coastlines-unsplit.db land_polygons
This gives another set of four files
simplified_land_polygons.dbf
simplified_land_polygons.prj
simplified_land_polygons.shp
simplified_land_polygons.shx
And this .shp looks like this:
Now when I copied these files to the /openstreetmap-carto/data/simplified-land-polygons-complete-3857/ directory, and re-ran renderd, I got a successful ghosting of the continents in the render (no screenshot, sorry, I forgot to take one).
Now I simply repeated the above, in every respect, but substituing my own rahet-planet.osm.pbf file for the ogf-planet.osm.pbf file above. I got the following shapefiles:
land_polygons.shp
simplified_land_polygons.shp
And these, copied to the appropriate osm-carto data directory locations, gives me the beautiful render you see now. [EDIT: Note that the view below of the Rahet planet is “live”, and therefore doesn’t match what shows in the screenshots above. I have moved in a different concept with my planet, and thus I have erased most of the continents and added different ones, and the planet is now called Arhet.]
I actually suspect this way that I did it is not the completely “right” way to do things. My main objective was to give the osm-carto shapefiles it would find satisfactory – it was not to try to reverse-engineer the actual OSM or OGF “coastline” processes.
There may be something kludgey about using the output of the second coastline run in the above two instances as the “simplified” shapefile, and this kludge might break if the Rahet or OGF planet coastlines were more complex, as they are for “Real Earth.” But I’ll save that problem for a future day.
A more immediate shapefile-based project would be to build north and south pole icecaps for Rahet, in parallel with the “Real Earth” Antarctic icesheets that I disabled for the current set-up. You can see where the icecaps belong – they are both sea-basins for the planet Rahet, but they are filled with glacial ice, cf. Antarctica’s probably below-sea-level central basin. And the planet Mahhal (my other planet) will require immense ice caps on both poles, down to about 45° latitude, since the planet is much colder than Earth or Rahet (tropical Mahhal has a climate similar to Alaska or Norway).
Happy mapping.
Music to map by: Café Tacuba, “El Borrego.”
A corporeal rebellion arose,
demon king goes cell by cell,
whom body could not expel.
[This is a cross-post from my other blog.]
[Update 20180923: continued from here]
The OSM “Rails Port” is now running on my server, and I have successfully connected to the api via JOSM and rebuilt my test-version of my planet, Rahet.
It took me an entire week of googling and meditating before I solved the port problem. Ultimately, I was looking in the wrong place for clear documentation about it – I was hoping someone would write about it from the perspective of Rails, but finally I found the documentation that made it possible on the Passenger website, buried in an example. There’s a line that belongs in the apache config file, “PassengerRuby /usr/bin/ruby2.3” (or whatever version).
And that made all the difference.
Here’s the link: MAP. [UPDATE 20210530: that link is broken – the test server is closed down. I am running a “live” planet for multiple users based on this original, at arhet.rent-a-planet.com]
So now you can look around. It’s just the “out-of-the-box” OpenStreetMap website (AKA Rails Port), with some minimal customization where I could find where to do it easily. I’ll continue working on that. I might actually disable the iD and Potlatch editing tools – I always use JOSM, and if it ever reaches a point where I’m allowing or inviting others to edit, I would make JOSM-use a prerequisite, I’m certain. JOSM, with its steep learning curve, seems like it would be a good way to “filter” people on the question of how serious they’re taking a project.
There are a number of features that don’t work. I would like to figure out a way to disable the user sign-up page. That’s a kind of vulnerability for the types of use I’m intending for this set-up. Meanwhile, I’ve disabled in a rather inelegant way by “breaking” the sign-up page (by changing its name inside the appropriate folder on the app/views/user path).
I’m happy.
I’ll write up how I figured out the coastline problem, tomorrow, and begin working on deciding what features to retain versus which to change in the Rails Port (i.e. think about customization).
[Update 20180923: continues here]
Music to map by: Run The Jewels, “Talk To Me.”
I had some weird dreams. I was in some kind of future-dystopian world where everything was subdivided into these enclosed hive-like spaces, but each space was the size of a city. So you could go from city to city via these doors in the hive walls. And most of the cities were run down, post-apocalyptic places, with gangs of wild children and insane people running things.
So I was trying to find the city where life was tolerable. It was like traversing a scaled-up version Borges' infinite library, but each room, instead of being a small study stocked with books, was a city. This might be a nice conceit for a novel. I'll get right on that.
What I'm listening to right now.
Cold, "Bleed."
Lyrics.
I'm feeling crossed, I take it inside
Burn up the pain, my thoughts are strange
Just like the things I used to know
Just like the tree that fell, I heard it
If art is still inside I feel it
I wanna' bleed, show the world all that I have inside
(I wanna' show you all the pain)
I wanna' scream, let the blood flow that keeps me alive
(I wanna' make you feel the same)
Take all these strings, they call my veins
Wrap them around, every fucking thing
Presence of people not for me
Well I must remain in tune forever
My love is music, I will marry melody
I wanna' bleed show the world all that I have inside
(I wanna' show you all the pain)
I wanna' scream let the blood flow that keeps me alive
(I wanna' make you feel the same)
I said
I wanna bleed
I wanna feel
(Show you all the pain)
I wanna scream
I wanna feel
(Make you feel the same)
Won't you let me take you for a ride
You can stop the world, try to change my mind
Won't you let me show you how it feels
You can stop the world, but you won't change me
I need music
I need music
I need music
To set me free
To let me bleed
[daily log: walking, 7km]
the brownian drift
of the gray bubbles of smog
scale to atmosphere
my nam yu no
alligaytur
i want tu ete
a mungki, shur,
or stoodents, yum,
in ther nise haus
but meenwile tho
i lik the maus
– This poem is in a completely new form, recently emergent from internet memedom, called "bredlik." In fact it's a pretty structured form, with requirements of rhyme, meter, theme and even a kind of anti-spelling convention. You can read about it here - linguists have been observing its development. As that summary notes, the misspellings are not meant to seem illiterate or childish, rather, they in fact somewhat emulate the fluid orthographies of Middle English. I would add that the deliberate misspelling also successfully conveys the orality of the poem in the context of the overwhelmingly textual medium of internet-based forums and chats. So I decided to make my own, about my classroom ubiquitous alligator character.
A 5th grade boy in a low-level class with other 5th grade boys managed a rather sophisticated exchange with me the other day.
The exchange began when I asked, "Are you ready, Mark?"
"Teacher, sorry. No," Mark complained.
I said, "You were supposed to memorize it. It's your turn."
Mark said, "Not ready." Gesturing with exaggerated politeness across the classroom at a classmate, he added, obsequiously, "John memorized very well."
John, being shy, made a look of alarm and grim consternation.
"Nice try, Mark," I laughed. "Close your book. Three, two, one… go! Start talking."
[daily log: walking, 7km]
I got home feeling so very tired.
I boiled some water to make tea.
The tea bag hung in a glass.
Hot water put off steam.
Time gnawed the edges.
In the water,
tendrils of
crimson
fall.
My friend Peter visits Seoul more frequently than I do. Which might not seem like such a notable thing, except that I live 25 km away and can go on a subway or bus, while he lives in Washington, DC.
So he stopped by on Sunday, using my apartment as a spot to leave his extra luggage so that he could be more mobile. I have no problem with that – he's been quite generous with me over the years, too.
This week has been pretty busy with work. Last night we had a 회식 (hweh-sik) after work, and on top of six classes in a back-to-back schedule, I was exhausted. We went to a galbi place, typical Korean fare, grilled at the table.
I am kind of tired and out-of-it today.
More later.
[daily log: walking, 7km]
[This is a cross-post from my other blog.]
I wanted to post a part 2 for my last post, about how I got the tileserver working. I was going to talk about coastlines. In fact, my tileserver IS working, but it feels a bit useless without the other half: the so-called Rails Port.
So I have become obsessed with trying to get the Rails Port running. And I keep running into problems. The fundamental problem is that I have never used Ruby (and/or “Ruby on Rails”) before. I don’t really understand it. It’s not a development environment I have any comfort with at all. I don’t really even get the overall model.
I can get a local version of the generic “openstreetmap-website” code running on port 3000 on my desktop. And I can get a similar “development” version running on my server. But I don’t know all the places I need to edit to get the Rails Port to “look at” my tile server and not the default OSM tileservers. And I don’t know what other files I need to customize to control e.g. users, site security, name presentation, etc.
I think I’m going to have to take a timeout on trying to set this up, and spend some time learning how to deploy a much simpler Ruby app on my server.
One bit that seems like it should be utterly trivial is how to get the application to present on port 80 (standard webpage) instead of port 3000 (Ruby’s default development port, I guess). I have installed Passenger for Apache and that’s how I can present the application on port 3000, but I guess Rails doesn’t cohabit well with other applications on Apache – e.g. the wiki, this blog, etc. So somehow… it has to get “wrapped” or “proxied” but the details of how to configure this are beyond my expertise.
I’m frustrated, so I’m going to take a break from this server stuff.
Music to map by: 박경애 – 곡예사의 첫사랑
The first day of spring
delivered snow with the rain
but the snow melted.
I don't have much to offer today. I was being obsessive with a computer thing, and didn't give myself time to think of a post for blogland. So here's this.
"When all you have is a database, everything looks like a segmentation problem."
I have not idea how to attribute this quote. It circulates online.
[daily log: walking, 4.5km]
ㅁ And I awoke: the air was viscous dust, athwart my jaw reclined some ghosts who had a blurry taste, frustration edible.
– a tercet in blank verse (iambic pentameter)
So these guys made a pop song in Italian complaining about people's failure to use the subjunctive properly. On the one hand, this is grammar peevery, and thus a linguist (such as I pretend to be on occasion) can't really be expected to approve. Grammar peevery is in fact diametrically opposed to rational, descriptive linguistics. Nevertheless, peevery can be entertaining, and it's funny to see Italians singing about grammar.
Cosa sto ascoltando al momento.
Lorenzo Baglioni, "Il Congiuntivo."
Testo.
[Intro]
Che io sia
Che io fossi
Che io sia stato
Ouooh
[Strofa 1]
Oggigiorno chi corteggia incontra sempre più difficoltà
Coi verbi al congiuntivo
Quindi è tempo di riaprire il manuale di grammatica, che è
Che è molto educativo
Gerundio, imperativo
Infinito, indicativo
Molti tempi e molte coniugazioni, ma
[Ritornello]
Il congiuntivo ha un ruolo distintivo
E si usa per eventi che non sono reali
È relativo a ciò che è soggettivo
A differenza di altri modi verbali
E adesso che lo sai anche tu
Non lo sbagli più
[Strofa 2]
Nel caso che il periodo sia della tipologia dell’irrealtà (si sa)
Ci vuole il congiuntivo
Tipo “Se tu avessi usato il congiuntivo trapassato
Con lei non sarebbe andata poi male”
Condizionale…
Segui la consecutio temporum
[Ritornello]
Il congiuntivo ha un ruolo distintivo
E si usa per eventi che non sono reali
È relativo a ciò che è soggettivo
A differenza di altri modi verbali
E adesso che lo sai anche tu
Non lo sbagli più
[Bridge]
E adesso ripassiamo un po' di verbi al congiuntivo:
Che io sia (presente)
Che io fossi (imperfetto)
Che io sia stato (passato)
Che fossi stato (trapassato)
Che io abbia (presente)
Che io avessi (imperfetto)
Che abbia avuto (passato)
Che avessi avuto (trapassato)
Che io sarei…
[Ritornello]
Il congiuntivo come ti dicevo
Si usa in questo tipo di costrutto sintattico
Dubitativo, quasi riflessivo
Descritto dal seguente esempio didattico
E adesso che lo sai anche tu
Non lo sbagli più
[daily log: walking, 7km]
[This is a cross-post from my other blog]
I thought I should put a discussion of how I did this, with much more detail, as I am sure there are other people out there in the world who might want to do something similar.
This is part 1. I’ll post part 2 later.
I wanted to be able to serve Openstreetmap-style map tiles of my own fictional planet, in the same way that the site OpenGeofiction does, but using my own data set.
This process of building a tileserver is separate from the job of setting up an Openstreetmap-style apidb database to be able to edit the data set using tools such as iD, Potlatch, or JOSM. I’m still working on that.
I deliberately set up my server on Ubuntu 16.04 (a flavor of Debian Linux) because I knew that OpenGeofiction runs in this environment. I’m not actually sure, but I assume Openstreetmap does too, though, given its scale, that may not be exactly the case, anymore – more likely it’s got a kind of customized, clustered Linux fork that has some genetic relationship to Ubuntu.
I thought it would therefore be easier to replicate the OpenGeofiction application stack.
Before starting this work, I had already installed MySQL and Apache and Mediawiki – except for Apache, however, these are not relevant to setting up a tileserver.
I had also already set up PostgreSQL (the preferred Openstreetmap database server), so the preliminary mentions of setting up this application were skipped.
Finally, using Apache’s sites-available config files and DNS, I had set up a subdomain on my server, tile.geofictician.net, to be the “outside address” for my tileserver. This will hopefully mean that if I ever decide to separate my tileserver from other things running on the same server, it will be somewhat easier to do.
Starting out, I mostly followed the steps and documentation at switch2osm.org’s detailed tutorial, here. Below, I refer to this page as S2OMBaTS (“switch2osm manually building a tile server”).
So I don’t see any need to repeat everything it says there. I just followed the steps given on that webpage exactly and religiously. What I’ll document are only the spots where I had to do something differently. These are my “deviations.”
I think this might be a bad solution from a security standpoint, but it’s the only one I could find that I understood and could get to work. PostgreSQL security is weird, to me. My DBA experience was entirely with SQLServer and Oracle, back in the day, and those databases’ security are integrated to OS security more tightly, I think. Similarly, MySQL seems to assume linkages between system users and database users, so security for the matched pairs of users are linked. But it seems like PostgreSQL doesn’t work that way.
osm2pgsql --database gis --create --slim --multi-geometry --keep-coastlines --hstore --verbose --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua --cache 2500 --cache-strategy optimized --number-processes 1 --style ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/south-korea-latest.osm.pbf
At this point, I reached the end of the S2OMBaTS tutorial.
I then had to customize things to load my own planet instead of a largely “naked earth” with South Korea well-mapped. The first step was easy enough. I just replaced the South Korea pbf extract with a pbf of my own planet, and re-ran the osm2pgsql step. I got the pbf extract of my planet by working with some kludges and with JOSM on my desktop machine. It was a “simplified” planet – just the continent outlines, a few cities, two countries with their admin_level=2 boundaries, and one tiny outlying island with lots of detail, which I borrowed from my city-state Tárrases at OpenGeofiction. It was composed as a kind of “test-planet” to keep things simple but hopefully test most of what I wanted to achieve in my tileserver.
Here’s the load script for that (essentially the same as used for South Korea, above).
osm2pgsql --database gis --create --slim --multi-geometry --keep-coastlines --hstore --verbose --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua --cache 2500 --cache-strategy optimized --number-processes 1 --style ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/gf-planet.osm.pbf
The problem, of course, is that if you run the render at this point, you get all the features of the new planet, but the continent outlines and the land-water distinction is “inherited” from earth. That’s because the mapnik style being used is referencing the shapefiles produced by and downloaded from Openstreetmap. The creators of the Openstreetmap software, including the OSM carto style, didn’t take into account the possibility that someone would try to use their software to show a map of somewhere that wasn’t planet Earth, and consequently, these need for these shapefiles is “hardcoded.” So the “Earth” shapefiles have to be substituted by alternate shapefiles extracted from the alternate planet dataset.
This was the hardest part for me. It took me more than a week to figure it all out. I’m not experienced with shapefiles, and don’t really understand them, and the process by which shapefiles get extracted from the OSM global dataset in a format that can be used by the openstreetmap-carto mapnik style is very poorly documented, online. So it was a lot of google-fu and experimentation, and downloading QGIS and teaching myself a bit about shapefiles, before I could get things working. It’s not clear to me that I really did it the right way. All I can say is that it seems to work.
The first steps I took were to try to simplify my task. I did this by chasing down the shapefile dependencies in the mapnik style sheet, and manually removing the ones that seemed less important. I did this mostly through trial and error.
The only file that needs to be edited to accomplish this simplification is the main mapnik xml file: <YOUR-PATH>/openstreetmap-carto/mapnik.xml. Bear in mind, though, that this file is the output of the carto engine (or whatever it’s called). By editing it, I have “broken” it – I won’t be able to upgrade my OSM carto style easily. But this is just a test run, right? I just wanted to get it to work.
So I edited the <YOUR-PATH>/openstreetmap-carto/mapnik.xml file and deleted some stuff. You have to be comfortable just going in and hacking around the giant xml file – just remember to only delete things at the same branch level of the tree structure so you don’t end up breaking the tree.
I removed the <Style></Style> sections that mentioned Antarctic icesheets – there were two. As things stand, my planet has no Antarctic icesheets, so why try to incorporate shapefiles supporting them?
Then, I eliminated the<Style></Style> section mentioning the <YOUR-PATH>/openstreetmap-carto/data/ne_110m_admin_0_boundary_lines_land directory, since these are land-boundaries for Earthly nations. I figured if I couldn’t see land-boundaries for my planet’s nations at low zooms, it would be no big deal. It’s not clear to me that this has been implemented on OpenGeofiction, either.
I also discovered that in fact, this file doesn’t even point to the <YOUR-PATH>/openstreetmap-carto/data/world_boundaries directory. So there was no need to worry about that one.
So that left me with two shapefiles I had to recreate for my own planet’s data:
<YOUR-PATH>/openstreetmap-carto/data/land-polygons-split-3857/land_polygons.shp and <YOUR-PATH>/openstreetmap-carto/data/simplified-land-polygons-complete-3857/simplified_land_polygons.shp.
Let’s just summarize by saying that this is what took so long. I had to figure out how to create shapefiles that the mapnik style would know what to do with, so that my continents would appear on the render. It took a lot of trial and error, but I’ll document what’s working for me, so far.
*** To be continued ***
[Update 20180923: Continues here]
Music to map by: Héctor Acosta, “Tu Veneno.“
ㅁ The sea rose up and swallowed the land immersing the empty spaces with a tide of blue pixels seething around houses always behind things under features pale blue dots here there here
– a nonnet
[This is a cross-post from my other blog (see previous blog entry)]
Here’s a live leaflet of my own tileserver with my own planet (stripped of detail because I want my database small as I test things). Welcome to Rahet. UPDATE, OCTOBER 2019: Being a dynamic window on the map, rather than a snapshot, means that since the “planet” shown is much changed, this view is not the view that existed when this blog post was written.
Here’s a view of Tárrases over at OGF on standard layer.
Here’s a view of Tárrases over at OGF on Topo layer. [UPDATE 20210530: The OGF Topo layer is no longer functioning.] [UPDATE2 20230315: The OGF Topo layer is once again functioning, and has been for over a year.]
That’s pretty cool.
Music to map by: Cold, “Bleed.”
[This is a cross-post from my other blog (see previous blog entry)]
A few weeks ago, I decided to just go ahead and try to replicate the “OpenGeofiction Stack” by building my own server.
So I shelled out ₩25000 KRW ($20 USD) a month for a low-end Linux server from one of the many companies that rent out cheap servers. It’s running Ubuntu 16.04.
I happened to have already bought, some years ago, the domain name ‘geofictician.net’, so I attached this name to my server, and I created some subdomains. I applied my moribund artistic skills and sketched up a little logo for the website, too. That’s also on this blog (at upper left). It’s a freehand drawing, but imitating some other images I looked at.
First I loaded the standard LAMP stack (MySQL and Apache), and I then installed mediawiki and configured. I made a kind of “clone” of the OGF wiki. I even uploaded some of the articles I’d deleted from the main site. I managed to get the MultiMaps extension fork that Thilo built running, so I can point those wiki articles at OGF.
The one thing I’m frustrated with, in the wiki, is that the email user utility was impossible to configure to work with my postfix install on the server. Hence, for now, I’ve got the wiki using my gmail account to send emails, which I think isn’t an ideal solution. Then again, it doesn’t really matter, for now, because it’s just me, using the wiki alone.
Anyway, I think I’ll use this wiki to write all the overwikification I’ve felt compelled to refrain from writing on the OGF wiki. Maybe I’ll build a bot and make stubs for ALL of my locales on the map (8000 stubs! Now that’s overwikification).
Next, I started building a tile server. This was pretty complicated, and I don’t consider the task complete. I did manage to upload an OSM file of a planet I started building using JOSM a few years back and that hasn’t ever been “rendered” before, though I’ve been drawing paper maps of parts of this planet since I was in middle school.
Finally, a few days ago, I was able to test the success of the tile render by connecting to the tileserver using JOSM from desktop. It was quite exciting to see my long-languishing planet, Rahet, rendered in JOSM, if only in the most skeletal of forms:
Earlier today, I installed this blog, using the standard wordpress package, and it went quite smoothly. Good-bye, bliki.
I’m currently working on getting the OSM “Rails Port” up and running. I just ran a test and got some errors, so I’ll have to troubleshoot those. But I feel like the end is in sight.
Music to map by: 마마무, “1cm의 자존심.”
I decided to start a separate blog on my new website.
There is a long history of me creating new "blogs" for one specific purpose or another. The longest-lived of my alternate blogs was the one I maintained for my job and students for several years. That blog still exists but it's largely dormant.
The reason for this new blog is that, although I don't mind sharing my geofiction activities here on this blog, I'm not sure how open I want to be about the rest of my life with fellow members of the geofiction community where I participate. That is, do they want to see or do they care to see my poetry, my ruminations of day-to-day classroom life, my oddball videos and proverb decipherments?
Since I think it's better to keep those things separate, I decided to make a separate blog. I also did it just to support the "technical unity" (if you will) of the website I've been constructing.
I may develop a habit of allowing the things I post on that other blog to appear here, but not vice versa. This blog would be the comprehensive "all Jared" blog, while that would be a kind of filtered version for the geofiction community.
Anyway, here's the blog (blog.geofictician.net), which currently has 4 posts, created over the weekend. Note that it seems like this blog will be fairly technical, representing the most abstruse aspects of my bizarre and embarrassing hobby, which might be termed "computational geofiction."
[daily log: walking, 7km]
In fact I don't much like crowds, they press in…
I prefer to be with clouds –
unholy shrouds.
Soyeon spoke to me today, after class ended. It had been one of the "CC" classes where I make the advanced middle-schoolers "teach" the class after preparing the materials, and I'd complimented her on having done a good job. She had. She's a natural teacher, maybe. And her spoken English, despite her twisted morass of underlying grammar problems, comes off as well-accented and mostly quite idiomatic.
She announced, somewhat proudly, "A boy asked me out yesterday."
Soyeon's in the seventh grade. I don't quite know what the typical "growing up" trajectory of a Korean teenager is like, but Soyeon far from typical. She is the most "American" Korean child I have ever known. I don't know how she got that way – she's never lived abroad and in fact has never visited an English-speaking country as far as I know. The closest she's gotten, I think, have been a few short trips to Thailand and Malaysia. I only mention that by way of saying that for all I know she's not particularly typical among her peers, but rather seems to be following a more Western route through adolescence, in which dating in middle school, if not universal, is certainly not viewed as unprecedented. I expect the more traditional Korean household would have none of this. But this is a society in rapid cultural transition, as usual, and individual families occupy quite distinct subcultures despite the broader homogeneity. Some families are utterly westernized, while others hove to a more traditionalist, even Confucian line.
Anyway, that's just background. I felt flattered that she offered this piece of news to me. It's demonstrative of a kind of trust. I've been her teacher for 4 years now, so I guess it's somewhat to be expected.
I said to her, "That's great. Do you like him?"
She nodded, and added, "He liked me."
"Liked you? Why do you say it in past tense? What happened?"
I was, pragmatically, wondering if this was grammatical mistake. Hence my question. But I was quite wrong. She demonstrated this, after a disproportionate delay in answering.
She looked at me slyly, and said, "Nothing happened. He liked me…." Another too-long pause. "Now he loves me." And an emphatic shrug, for punctuation.
Ah. It was a joke.
"Well that's fast," I said, as neutrally as possible.
"Uh huh," she said, with a near native-English-speaking teenage tonality. And walked away, playing some game on her smartphone.
[daily log: walking, 7.5km]
