• More Database Geekery

    Nothing new and interesting to report for the past few days. I’ve been continuing to work on the same knitting project. The old cat is getting older and is starting to have the types of issues that you would not be surprised to hear about in a 15-year-old cat.

    But, never mind the boring stuff. Yesterday I made some materialized views in the database, and they were awesome!

    I’ve mentioned this before, but the production database for running the site was primarily designed to run the site and do all the things that need to be done in order to keep things running. Most of the interactions with the database are like, “This user did that thing,” or “Find the things for this user.” On the other hand, the analysis queries are more along the lines of, “Find all the users in these categories who did those things during that timeframe and calculate the average number of points that they scored under certain circumstances.” The latter tend to be slower. They can be really slow. Far too slow to be allowed to run on the production server.

    Most of my queries do follow certain patterns, though. I join together the same tables on a regular basis, and these joins are pretty time-consuming. A materialized view lets me do all my expensive joins and other slow operations once in the middle of the night and then save the output in a table-like structure (it is so table-like that I can even define indices on it!), and then at run-time, my dashboards can query the materialized view. Queries that used to take minutes to run will now execute in under a second!

    It also makes my queries easier to read because I am not joining together a rat’s nest of tables in order to get the information that I am looking for.

    More updates to come. The first legwarmer is almost done! The cat is going to try to take pills, so maybe I’ll have some interesting scratches and bite marks to show off.


  • What's So Special About Schenectady?

    Yesterday afternoon and last night I spent more time than is strictly necessary on figuring out how to use the Census’s GIS API to return GeoJSON so that I could draw maps on web pages.

    Long story short, my map of Schenectady County was perfect. The code for automatically scaling the bounding box was flawless, and my map would fill the entire SVG element. On the other hand, the map of San Diego County was wrong, wrong, wrong, wrong, wrong. Sure, it was centered in the box, but the map was tiny. This happened for every non-Schenectady County that I tried. Albany County? Fail. Broome County? Fail. Oneida County? Fail.

    Why, why, why is the only county that works the county that I grew up in?

    I figured maybe something was being cached somewhere, since most of the original just-get-it-to-work-in-the-first-place-darnit code was written with Schenectady County hard-coded into the script. Search for the letter combination “sc” everywhere in the file; it only shows up in script tags. Search for the magic numbers 36 and 93. Nothing.

    Hard reload. Clear cookies, cache, and local storage. Try a different browser. Download a new browser and use igcognito mode. Still broken.

    I opened the other laptop and opened the pages over my local network. Still broken.

    Was my router caching the responses from the API? I put the files In The Cloud (maybe they are still here in /maps/foo.html and /maps/bar.html) turned off the wifi on my phone and viewed them over my phone’s data connection. Still broken.

    There’s got to be something about Schenectady.

    Outside of New York City, Schenectady County is the smallest county in New York (geographically). And not a lot of people live there, so there are not a lot of census tracts or census block groups or other people-containing geographic features. Schenectady County is small enough and sparsely populated enough that the API is able to return the GeoJSON without timing out or throwing an error. The other counties were all larger and/or more populous, and the error from the server was being read in by the map-scaling code.

    Next up, I picked a county in Rhode Island. Works perfectly.


  • Knitting Update

    Knitting update! Current project is now at the stage of “more than a third of a legwarmer very unusual upside-down sock.”

    Knitting project


  • More Quick Updates

    1. Sorry. Another list post.

    2. WE DID THE TAXES BEFORE THEY WERE DUE. Ended up owing over three thousand bucks, though. Note that I did not even need to actually threaten that I was not going to renew the HBO subscription until the taxes were done.

    3. Now I need to renew the HBO subscription. Yes, yes I did look up the dates for Season 8 just to see if I could save fifteen bucks by waiting to sign up a month before the last episode so that we could watch all six episodes during one month. But then I was like, “eff it, I just spent three grand on putting kids in cages and whatever other nonsense things my federal government does. I should indulge myself in a TWO MONTH subscription to HBO so that I can see all the episodes on the same day as everyone else.”

    4. Secretly, I am pretending that the three grand (ok, more like $3500, but we are getting money back from the state) (plus the many other thousands of dollars that we had withheld from our paychecks) is going to the Department of Commerce and the Census API. Maybe some of it to the FAA.

    5. Cover letter update: Someone sent in an application with an eight-word cover letter. This “cover letter” was not actually a complete sentence. I moved the application on to the next step anyway, as an eight-word cover letter is still an existing cover letter.

    6. Both cats get to go to the veterinarian on Saturday! Woo! I already got the cat carriers out because the stupid cat hides under the couch when I first get the cat carrier out. But after it has been sitting around the living room for a few days, she will start to sleep in it.


  • Quick Updates

    1. Got an MRI of my head on Saturday. Got the results this morning: Nothing of note.

    2. The claim from the imaging center is that they do MRIs on weekends as a covenience to patients. I suspect it is because the machine brings in a lot of money. Sort of like how airlines try really hard to have all their planes in the air as much as possible.

    3. Although I was kind of suprised that a radiologist had time to read the MRI over the weekend. Since the imaging center is affiliated with a hospital chain, maybe the radiologists at the hospital read these MRIs when things are quiet at the hospital? As this is a different hospital system from the one where one of my former students is a radiology resident, I don’t know anyone who I can ask.

    4. Finally wised up on the third attempt to start my knitting project and put a lifeline at a key transition point. Not even two rounds past the lifeline, it looks like I am going to need it. How many times can this yarn tolerate being ripped out and reknitted? WE SHALL SEE.

    5. Also this weekend, looked through the Michelle Memorial Archives as part of an effort to show someone I know how actually and totally crazy they are. Also half-heartedly considering a book (the word “book” being construed very loosely) about the contents of the Michelle Memorial Archives.

    6. While looking at a publication in which Michelle published an essay, I discovered that a bead crocheter that I know also published an essay in that issue.

    7. Had dinner with a relative on Sunday.

    8. Work today has been a mix of catching up on all the things that came in over the weekend and trying to decide how hard to try to do something that one of my colleagues muses might be impossible. You see, there are two databases, and they have related data in similar forms, but they are not the same. Is there a way to make the reporting from one database match with the interface provided from the other database? Currently unclear.


  • Starting a Conspiracy Theory

    As a resident of our new, post-truth universe, I would like to start a totally untrue rumor/conpiracy theory.

    I am going to assert that Andrew Yang does not exist.

    As you might recall, I am from Schenectady. Andrew Yang is allegedly from Schenectady. No one I know knows this Andrew Yang. (We know a different Andrew Yang who is maybe three years older than the one who is running for president.) Specifically, my brother does not know Andrew Yang despite being from the same hometown, roughly one grade apart, and going to the same college.

    And, yes, Andrew Yang went to prep school. But other people who I know went to the same prep school! And they do not know him, either! No one remembers him from middle school!

    This would be the most amazing conspiracy theory in the whole world if Roxane Gay (who also went to that prep school and is also roughly the same age as me, my brother, and this alleged “Andrew Yang” character) had no recollection of him either.

    Part of me sort of hopes that my conspiracy theory is true. I’ve had enough of some of the real people running for president. An imaginary candidate might be a good break from reality.


  • The Gaping Maw of Content

    We have brought this upon ourselves at work: Not only do we provide all of our usual educational content, but we now also have a digital media thing going on. There is a podcast (which, like all podcasts, wants everyone to rate and review!). There is what is being called an online magazine, but which I think of as a company blog.

    Now that we have these things, they need to be fed and cared for. And since they are official publications, they need to stick to the official branding guidelines. I was asked if I wanted to write something for the online magazine. I do not. My writing interests have only a minimal intersection with the goals of this publication. And someone else already wrote about knitting.

    My one contribution was to make life a little bit more difficult for the staff setting this whole thing up. I pointed out that .news is a top level domain and that we should buy the .news domain corresponding to this product and point it at the online magazine.

    Plus, I already have two blogs that I can barely keep up with. (The other one is very pretend and mostly short comments about Pokémon Go.)


subscribe via RSS