Caveat: not not choose

"I am my choices. I cannot not choose. If I do not choose, that is still a choice. If faced with inevitable circumstances, we still choose how we are in those circumstances." – While this quote is widely attributed (as an English translation) to Jean-Paul Sartre, I can't seem to validate it in any kind of original French-language text. Certainly he said something similar, though.

[daily log: walking, 2km]

Caveat: Poem #608

Kiamon never once thought on her fate
Grimly she battled to push down her hate
Hoping perhaps to at last find her goal
Kiamon willingly gave up her soul.

…Recently I tried something new. As some of you know, I have a rather wide set of "novels in progress," none of which actually progress, much. I'm bad at these wider, longer-scale projects. So I decided to take this slightly more successful short poem-a-day concept and "hijack" it for the novel thing. I have been writing little "character-building" quatrains, where I try to encapsulate some aspect of a story's character. This is one of those. In general, don't be surprised to see the names of fictional beings begin to populate some of my poems.

[daily log: walking, 1km]

Caveat: No fiction was it of the antique age

The Faëry Chasm

No fiction was it of the antique age:
A sky-blue stone, within this sunless cleft,
Is of the very footmarks unbereft
Which tiny Elves impressed; – on that smooth stage
Dancing with all their brilliant equipage
In secret revels – haply after theft
Of some sweet Babe – Flower stolen, and coarse Weed left
For the distracted Mother to assuage
Her grief with, as she might! – But, where, oh! where
Is traceable a vestige of the notes
That ruled those dances wild in character? –
Deep underground? Or in the upper air,
On the shrill wind of midnight? or where floats
O'er twilight fields the autumnal gossamer?

– William Wordsworth (English poet, 1770-1850)

[daily log: walking, 7km]

Caveat: rich in entropy

We live in a weird era. Entropy has become a kind of commodity in and of itself.

In all this messing around with my new server… with trying out new things and tinkering with it all… well, of course I have to educate myself a bit about server security. It's a big, bad world out there, and if I'm going to be running a server that's publicly visible on the internet (offering up webpages, etc.) the little machine will be lonely and vulnerable, and I have to think about how to protect it from bots and blackhats.

In the field of network security, one thing that comes up is that you have to have some fundamental understanding of the types of cryptography used these days to secure systems. There's a whole infrastructure around generating "secure" public and private keys that computers hoard and exchange with one another to authenticate themselves. I really DON'T understand this, but I wade through the documentation as I e.g. try to set up a certificate authority on my server, because some of the things I'm running there apparently require it. I run the commands they tell me to, and hopefully my little server is sorta secure. But who knows.

I was fascinated to learn, however, about a thing that is used in crypto key generation on computers: system "entropy."

On one site I was looking at, there was a discussion about the fact that virtual machines (the sorts you rent from big companies to run cheap little servers, as I have done) have extremely low "available entropy" while your typical crummy desktop has very high "available entropy" – therefore when I generate my keys, I should do so on my desktop, not my server – I can upload the generated keys to my server later.

I think it's kind of a funny concept. The mass-produced, cookie-cutter, high quality, reliable servers found on the giant server farms are lacking in a certain commodity that they desperately need for their security: entropy. So the admins have to go out to their desktops to get the entropy they need. I sit here and I listen to my cruddy, 7 year old Jooyontech Korean PC-clone desktop, with its perpetually failing CPU fan groaning intermittently and the weird system noises filtering though the sound channel onto my speakers, and I can rest assured that that's all part and parcel of having lots and lots of good, tasty entropy that I can feed to my server in the form of so many sweet, generated security keys.

One site I was reading said that typical desktop entropy should be around 2000 (in whatever units entropy is measured with…).

Out of curiosity, I plugged in the Linux command that would tell me my system's entropy. I got 3770. Wow! I'm rich! … in entropy, anyway.

Meanwhile, my server, a virtual machine in some well-air-conditioned server farm facility across the Pacific in California, manages only 325 units of entropy. So sad. The chaos-poor, withered fruits of conformity.

[daily log: walking, 7km]

Caveat: Thealligu +or wins

My 5th grade student John recently started at Karma. He has never studied English before, so in the context of Korean English education, he's a rather "late starter." It's hard to place him at a place like Karma, because classes at his grade level don't really have "beginner status". Still, I was very proud when he produced the following comic during an exercise in class. He carefully asked (in Korean) how to say each thing in English, and I wrote the translations on the board. Then he copied the letters (he hasn't even mastered the alphabet) into his comic. I thought his characters (an imitation of my alligator character, along with some lion character he created), while quite rudimentary, were cute, too. He might have some artistic talent, anyway.

picture

[daily log: walking, 7km]

Caveat: A more technical summary of how I built my tileserver – part 2 (cross-post)

[The following is a direct cross-post from my other blog – just so you don't think I'm doing nothing with my free time, these days.]

The objective

I started discussing the coastline shapefile problem in first post.

Early on, I found the tool called QGIS Browser and installed it on my desktop. I used this to examine the shapefiles I was creating.

The first step was to look at the "real Earth" OpenStreetMap-provided shapefiles I was trying to emulate – the two mentioned in my previous post:
/openstreetmap-carto/data/land-polygons-split-3857/land_polygons.shp
and
/openstreetmap-carto/data/simplified-land-polygons-complete-3857/simplified_land_polygons.shp

Here are screenshots of each one.

First, land_polygons.shp

And here is simplified_land_polygons.shp

The structure is pretty straightforward, but – how do I make these? Where do they come from? – aside from the non-useful explanation found most places, which is that "OpenStreetMap generates them from the data".

The coastline problem

The way that the shapefiles are generated for OpenStreetMap are not well documented. But after looking around, I found a tool on github (a software code-sharing site) developed by one of the OpenStreetMap gods, Jochen Topf. It is called osmcoastline, which seemed to be the right way to proceed. I imagined (though I don't actually know this) that this is what's being used on the OpenStreetMap website to generate these shapefiles.

The first thing I had to do was get the osmcoastline tool working, which was not trivial, because apparently a lot of its components and prerequisites are not in their most up-to-date or compatible versions in the Ubuntu default repositories.

So for many of the most important parts, I needed to download each chunk of source code and compile them, one by one. Here is the detailed breakdown (in case someone is trying to find how to do this).

Installing osmcoastline

I followed the instructions on the github site (https://github.com/osmcode/osmcoastline), but had to compile my own version of things more than that site implied was necessary. Note that there are other prerequisites not listed below here, like git, which can be gotten via standard repositories, e.g. using apt-get on Ubuntu. What follows took about a day to figure out, with many false starts and incompatible installs, uninstalls, re-installs, as I figured out which things needed up-to-date versions and which could use the repository versions.

I have a directory called src on my user home directory on my server. So first I went there to do all this work.
cd ~/src

I added these utilities:
sudo apt-get install libboost-program-options-dev libboost-dev libprotobuf-dev protobuf-compiler libosmpbf-dev zlib1g-dev libexpat1-dev cmake libutfcpp-dev zlib1g-dev libgdal1-dev libgeos-dev sqlite3 pandoc

I got the most up-to-date version of libosmium (which did not require compile because it's just a collections of headers):
git clone https://github.com/osmcode/libosmium.git

Then I had to install protozero (and the repository version seemed incompatible, so I had to go back, uninstall, and compile my own, like this):

Git the files…
git clone https://github.com/mapbox/protozero.git
Then compile it…
cd protozero

mkdir build

cd build

cmake ..

make

ctest

sudo make install

I had to do the same for the osmium toolset:

Git the files…
git clone https://github.com/osmcode/osmium-tool.git
Then compile it…
cd osmium-tool

mkdir build

cd build

cmake ..

make

That takes care of the prerequisites. Installing in the tool itself is the same process, though:

Git the files…
git clone https://github.com/osmcode/osmcoastline.git
Then compile it…
cd osmcoastline

mkdir build

cd build

cmake ..

make

I had to test the osmcoastline tool:
./runtest.sh

Using osmcoastline for OGF data

So now I had to try it out. Bear in mind that each command line below took several hours (even days!) of trial and error before I figured out what I was doing. So what you see looks simple, but it took me a long time to figure out. In each case, after making the shapefile, I would copy it over to my desktop and look at it, using the QGIS browser tool. This helped me get an in intuitive, visual feel of what it was I was creating, and helped me understand the processes better. I'll put in screenshots of the resulting QGIS Browser shapefile preview.

To start out, I decided to use the OGF (OpenGeofiction) planet file. This was because the shapefiles were clearly being successfully generated on the site, but I didn't have access to them – so it seemed the right level of challenge to try to replicate the process. It took me a few days to figure it out. Here's what I found.

Just running the osmcoastline tool in what you might call "regular" mode (but with the right projection!) got me a set of files that looked right. Here's the command line invocation I used:
YOUR-PATH/src/osmcoastline/build/src/osmcoastline --verbose --srs=3857 --overwrite --output-lines --output-polygons=both --output-rings --output-database "YOUR-PATH/data/ogf-coastlines-split.db" "YOUR-PATH/data/ogf-planet.osm.pbf"

Then you turn the mini self-contained database file into a shapefile set using a utility called ogr2ogr (I guess part of osmium?):
ogr2ogr -f "ESRI Shapefile" land_polygons.shp ogf-coastlines-split.db land_polygons

This gives a set of four files
land_polygons.dbf

land_polygons.prj

land_polygons.shp

land_polygons.shx

Here is a view of the .shp file in the QGIS Browser. Looks good.

I copied these files into the /openstreetmap-carto/data/land-polygons-split-3857/ directory, and I tried to run renderd. This alone doesn't show the expected "ghost" of the OGF continenents, though. Clearly the simplified_land_polygons.shp are also needed.

So now I experimented, and finally got something "simplified" by running the following command line invocation (note setting of –max-points=0, which apparently prevents the fractal-like subdivision of complex shapes – technically this is not really "simplified" but the end result seemed to satisfy the osm-carto requirements):
YOUR-PATH/src/osmcoastline/build/src/osmcoastline --verbose --srs=3857 --overwrite --output-lines --output-rings --max-points=0 --output-database "YOUR-PATH/data/ogf-coastlines-unsplit.db" "YOUR-PATH/data/ogf-planet.osm.pbf"

Again, make the database file into shapefiles:
ogr2ogr -f "ESRI Shapefile" simplified_land_polygons.shp ogf-coastlines-unsplit.db land_polygons

This gives another set of four files
simplified_land_polygons.dbf

simplified_land_polygons.prj

simplified_land_polygons.shp

simplified_land_polygons.shx

And this .shp looks like this:

Now when I copied these files to the /openstreetmap-carto/data/simplified-land-polygons-complete-3857/ directory, and re-ran renderd, I got a successful ghosting of the continents in the render (no screenshot, sorry, I forgot to take one).

Using osmcoastline for my own data

Now I simply repeated the above, in every respect, but substituing my own rahet-planet.osm.pbf file for the ogf-planet.osm.pbf file above. I got the following shapefiles:

land_polygons.shp

simplified_land_polygons.shp

And these, copied to the appropriate osm-carto data directory locations, gives me the beautiful render you see now.

[Unfortunately, the leaflet script kludge that I once had working before for this typepad-hosted blog doesn't seem to be working (not allowing me to add raw javascript to my entry), so I can't replicate the slippy-map window here. But the slippy-map shows up in the original blog post that this blog post is a copy of, and of course you can see the slippy map in all its glory at test.geofictician.net (this is a test instance and may be subject to link-rot, however).]

I actually suspect this way that I did it is not the completely "right" way to do things. My main objective was to give the osm-carto shapefiles it would find satisfactory – it was not to try to reverse-engineer the actual OSM or OGF "coastline" processes.

There may be something kludgey about using the output of the second coastline run in the above two instances as the "simplified" shapefile, and this kludge might break if the Rahet or OGF planet coastlines were more complex, as they are for "Real Earth." But I'll save that problem for a future day.

A more immediate shapefile-based project would be to build north and south pole icecaps for Rahet, in parallel with the "Real Earth" Antarctic icesheets that I disabled for the current set-up. You can see where the icecaps belong – they are both sea-basins for the planet Rahet, but they are filled with glacial ice, cf. Antarctica's probably below-sea-level central basin. And the planet Mahhal (my other planet) will require immense ice caps on both poles, down to about 45° latitude, since the planet is much colder than Earth or Rahet (tropical Mahhal has a climate similar to Alaska or Norway).

Happy mapping.

Music to map by: Café Tacuba, "El Borrego."

[daily log: walking, 7km]

Caveat: my thoughts are strange

I had some weird dreams. I was in some kind of future-dystopian world where everything was subdivided into these enclosed hive-like spaces, but each space was the size of a city. So you could go from city to city via these doors in the hive walls. And most of the cities were run down, post-apocalyptic places, with gangs of wild children and insane people running things.

So I was trying to find the city where life was tolerable. It was like traversing a scaled-up version Borges' infinite library, but each room, instead of being a small study stocked with books, was a city. This might be a nice conceit for a novel. I'll get right on that.


What I'm listening to right now.

Cold, "Bleed."

Lyrics.

I'm feeling crossed, I take it inside
Burn up the pain, my thoughts are strange
Just like the things I used to know
Just like the tree that fell, I heard it
If art is still inside I feel it

I wanna' bleed, show the world all that I have inside
(I wanna' show you all the pain)
I wanna' scream, let the blood flow that keeps me alive
(I wanna' make you feel the same)

Take all these strings, they call my veins
Wrap them around, every fucking thing
Presence of people not for me
Well I must remain in tune forever
My love is music, I will marry melody

I wanna' bleed show the world all that I have inside
(I wanna' show you all the pain)
I wanna' scream let the blood flow that keeps me alive
(I wanna' make you feel the same)
I said
I wanna bleed
I wanna feel
(Show you all the pain)
I wanna scream
I wanna feel
(Make you feel the same)

Won't you let me take you for a ride
You can stop the world, try to change my mind
Won't you let me show you how it feels
You can stop the world, but you won't change me
I need music
I need music
I need music
To set me free
To let me bleed

[daily log: walking, 7km]

Caveat: Poem #600

my nam yu no
alligaytur
i want tu ete
a mungki, shur,
or stoodents, yum,
in ther nise haus
but meenwile tho
i lik the maus

– This poem is in a completely new form, recently emergent from internet memedom, called "bredlik." In fact it's a pretty structured form, with requirements of rhyme, meter, theme and even a kind of anti-spelling convention. You can read about it here - linguists have been observing its development. As that summary notes, the misspellings are not meant to seem illiterate or childish, rather, they in fact somewhat emulate the fluid orthographies of Middle English. I would add that the deliberate misspelling also successfully conveys the orality of the poem in the context of the overwhelmingly textual medium of internet-based forums and chats. So I decided to make my own, about my classroom ubiquitous alligator character.

Caveat: passing the buck

A 5th grade boy in a low-level class with other 5th grade boys managed a rather sophisticated exchange with me the other day.

The exchange began when I asked, "Are you ready, Mark?"

"Teacher, sorry. No," Mark complained.

I said, "You were supposed to memorize it. It's your turn."

Mark said, "Not ready." Gesturing with exaggerated politeness across the classroom at a classmate, he added, obsequiously, "John memorized very well."

John, being shy, made a look of alarm and grim consternation.

"Nice try, Mark," I laughed. "Close your book. Three, two, one… go! Start talking."

[daily log: walking, 7km]

Caveat: how often do you visit Seoul?

picture

My friend Peter visits Seoul more frequently than I do. Which might not seem like such a notable thing, except that I live 25 km away and can go on a subway or bus, while he lives in Washington, DC. 

So he stopped by on Sunday, using my apartment as a spot to leave his extra luggage so that he could be more mobile. I have no problem with that – he's been quite generous with me over the years, too.

This week has been pretty busy with work. Last night we had a 회식 (hweh-sik) after work, and on top of six classes in a back-to-back schedule, I was exhausted. We went to a galbi place, typical Korean fare, grilled at the table.

I am kind of tired and out-of-it today.

More later.

[daily log: walking, 7km]