Datastore (Google App Engine) does not let you lock the database. So how do you map across every element in Datastore? Here is a very rough and tumble solution to that problem.

Usage

Assume you have a ndb.Model named Widget

Begin pagination using the following (note that Widget is case sensitive)

curl https://example-domain.appspot.com/paginate/Widget
# => {'keys': [<URLSAFEKEYS>], 'cursor': <CURSOR_VALUE>}`.

Use <CURSOR_VALUE> to continue pagination across all Widget's

curl https://example-domain.appspot.com/paginate/Widget/<CURSOR_VALUE>

Code

Him: "Another museum on it's way..."

Me: "What?"

Him: "I mean, if you had an atomic aircraft, where would you park it?"

Me: ... <few seconds of though> ... "Area 51?"

Him: Momentary shock registers on his face, skuddles/crab walks away.

Just a random conversation with some of the fairly obviously tweaking tenants in my apartment...

I have never been much of an academic. In high school, all my friends were accelerated 1 year and allowed to test out of algebra 2; my scores were too low to allow me to skip it. Not wanting to be left behind, I resolved to take algebra II over summer.

That was a mistake.

That Summer I took algebra II with some laughably bad students at an inner city school. It was surreal. We would spend 4 hours a day in class, doing almost no real learning. I believe one kid was actively jerking off in the back of the room. One girl was pregnant and was taking life advice from another teen mother. By the end, we had covered maybe 3 out of 15 chapters. It was a complete failure. Somehow everyone who showed up passed. I later heard that Texas had so many "super seniors" clogging up the system that they created summer classes like this which were guaranteed passes; the goal was not education, the goal was to move people out of the system.

Like an idiot, I then took my "pass" in algebra II to my own school. I was happy, I could stay with my friends! Mathematically, things quickly fell apart. I realized that I was in over my head. The prospect of admitting that I couldn't hack it was even worse than my previous fear of being left behind (teen logic at its finest). For the next 3 years, rather than understand anything, I built elaborate workarounds to avoid exposing my ignorance. I didn't know how to actually solve most problems, I did however understand that my TI-83+ calculator could approximate the solution to most everything. With a little bit of flourish I could provide something that looked like I worked through a problem, despite the fact that the only thing I really had was a solution and an understanding of my calculator.

I graduated missing around 4 years of high school mathematics.

I feel like failing to understand some of the basic concepts in Algebra II substantially retarded my progress in mathematics. I lacked the basics that were necessary to move forward.

Mathematical Learning

The plural of anecdote is not data. With that said, this video resonates with me. I like the idea of mathematics being a small graph of deeply connected nodes. Failing to understand one of the axiomatic/deeply-networked nodes can make understanding the interconnected nodes very difficult.

Contrast this to history with its much larger number of nodes. These nodes are interconnected, but not as dependent on each other. They might provide context or contrast relative to one another, but they are not required for understanding.

I though this was an interesting point.

I found the following intriguing because it unified the notion of habits and the notion of self. I have always viewed my habits as external things, there is me and then then there are my habits. Habits are something external to me. "I am attempting to practice the guitar every day." "I am attempting to give up eating dessert every meal." I view habits as something outside of myself; something I do with willpower.

It is a powerful shift in perception to consider that you are a collection of habits. That the emergent notion of you is a complex interweaving of habits; some so fundamental that you have "subsumed" them into your definition of self.

I find it comforting to consider that all reactions to the world are just a matter of habits. Skills are habits that help you react properly to the world. Addictions are habits that have become too ingrained and lack nuance. All problems can be tackled by thinking of them as habits that need to be internalized (or broken) to allow you to get to your desired outcome.

A "what if" study of a fully connected world.

The central premise of 2017's "The Circle" is that the world has been fully connected through a network of always on, always watching, tiny, cheap, inocuous cameras.

It is technically sci-fi, though all the elements I observed are more than reasonable. More a matter of minitiuarization and falling hardware prices than actual technological breakthroughs.

The Good

I am very forgiving of things that cause me to change my perception, or open me to a new way of thinking. Did this movie have flaws? Absolutely. I'll get to those later. Let's talk about what it did well.

Use of a character as a proxy for popular culture

Emma Watson went through the stages of acceptance of an omnipotent society in an extremely collapsed timeline. What she did in a matter of months most people in developed nations are still aclimatizing to.

At one point Tom Hanks turns to her and gets her to "admit" that having experiences that you do not share with others is a form of theft. A form of emotional hording. At this moment in history, such an idea seems ludicrous. Almost perverse. Only celebrities and politicians have no right to privacy? Right?

I don't know.

I am not sure people 15 years from now will agree. How would someone from 15 years ago feel about selfie sticks? Snapchats of your dinner? Tweeting by politicians. Sexting. Facebook friends. Online Dating. Etc. Probably seems pretty ludicrous. Inhuman. Disconnected. Shallow. And yet, here we are.

Accounting the cost and benefits of omnipotence

Yeah, a lot of great things could occur. From the trivial to the fundamental.

• Immediate reactions as machines watching our feeds recognize violence, injury, and danger and automatically notify and dispatch emergency responders.

• Automatically and constantly dealing with fundamental weaknesses in society. Making sure that people don't get too isolated; too lonely. That people who exhibit sighs of mental disorders are regularly visited by health care professionals. Checking in on everyone with the (literal) persistence of a machine.

• Violent crime does not make sense. Only physical crime that would still exist would be crimes of passion. Crimes that law enforcement was incapable of negating due to being sudden and unexpected. Although tragic, when a violent crime occurs, it would almost always be evident what occured and who was responsible.

Ask yourself? How much would you be willing to give up for such a world?

Open Goverment

Movie makes a big deal about having a open (probably puppet) politician.

Prisons

Here is one that I fully agree with. Prisons should be places that have cameras literally everywhere. In a high risk environment like a prison, I would happily trade my privacy for the knowledge that I am not going to be raped or assaulted.

Police Officers

This is already occuring. I forsee that in the next decade, the cost of insuring a police officer who refuses to wear (and use) a full body camera will be so high that departments will require police officers to have them running at all times.

Politicians

Heh, I don't know. It would be interesting. Could most of the goverment be entirely opened? Would we allow "black boxes" like the CIA, FBI, NSA, etc? Would it turn out that backroom negotiations by our nations actually do benefit citizens more than they hurt? Would it cut corruption? Hard to say.

The Bad

• What was John Boyega constantly looking at on his phone?
• Why did John Boyega show her the basement? What was the significance of mentioning the water at the end of the tunnel?
• What the hell was John Boyega's problem? Why did he seem to have all this power to do all these things, and he literally just stood around the entire movie doing... what? Waiting for Emma Watson to make a decision about the future of our "omnipotent" distopia/utopia? It made little sense.
• Software engineers tend to be pretty opinionated. There would be a lot of dissent about some of the things that "Circle" was doing. It was a little unrealistic how "united" (cult like) the "Circle" corporate family was.
• Ellar Coltrane was a rather odd inclusion. From the scene where he comes to the Circle campus to complain about being a "deer killer" onward, it just didn't fit.
• The Ellar Coltrane rising action and denouement was pretty lame.
• Emma Watson's transformation from being a skeptic of the omnipotence to a supporter did not really feel earned. Might have played a little more to why she is putting away her previous skepticism and what particular benefits she thinks a fully connected society will bring.
• Honestly, too many ideas, not enough time to give each of them their own merrit. This is a difficult and multifaceted question with most of its sharp corners filled down.

Conclusion (spoil the last scene in the movie here)

Yes, it has some serious flaws. But it gives you things to think about, and it honestly resolved the ending in a rather novel way. Rather than fighting the system, Emma Watson instead fully opens and embraces it. Basically making the case that if such a system exist, it needs to be placed under open rather closed controlers.

If you are going to have a system that is going to audit the world, then you must audit everyone in control of that system.

I went to clojurewest 2017 in Portland, Oregon. Good conference, lots of good information. Here are my observations, my notes, and a few pictures.

The Good

The conference was well managed. The talks were mosty interesting. The community continues to be friendly, open, and inclusive.

The Bad

I wish there had been a few more "This is something I worked on that is very cool" type presentations, but you can't force those. Maybe there just wasn't as much cool stuff in 2017 compared to previous years. I guess every year can't be a "Clojure now compiles to javascript" or "core.async, use it" kind of year.

The City

Portland is an interesting city. I didn't actually realize how small and walkable the core of the city is. In San Francisco, I have practially forgotten what it is to see families with kids on the street. It is certainly a nice place to visit. I liked the eating, the literary/craft tradition, the beer, and the people. It did have a number of younger homeless (kind of reminded me of Berkeley). It did have the feeling of a place that gets a lot of tourist (that can be tiring when you live there). I could see myself being very happy there, but I haven't heard a whole lot about the tech scene there. The public transportation is excellent and the lack of traffic is a pleasure. I don't know if it is a place to "see and be seen" but it certainly felt like a place to "live and let live". Kind of like the kind of "weird" that Austin aspires to.

The Pictures

Clojure/West 2017

The Notes

Core.Async in Use - Timothy Balridge - core.async

• Count is O(n) on a concat, as concat is lazy and only closes over two collections
• Recommends understanding transducers if you are going to continue using core.async.
• If you are going to use async code, then consider writing a framework such that the asynchronous code is hidden from the user.
• High Level:
  • Keep userspace code pure
  • Move complexity out of user space
  • Make dependencies explicit
  • Leverage this for easier testing
  • Use core.async to enable cleaner abstraction, not as an end in itself.

With Great Composition Comes Great Responsibility - Elizabeth Engelman

• Datomic data is immutable
• Datomic is ACID
• Datomic is very flexible
• You can of course just concat if you want to put where clauses together

Using spec to Transparently Replace a Legacy System - Daniel Solano Gómez

• Capture a day’s worth GET and generate specs for them. Now you can generate inputs used on your system automatically.
• Maybe this is an example of actually being able to do full rewrites. If you have a endpoint system than you should be able to replace it by specing it out.
• In a odd way, kind of another argument to keep the API of your old customers live. Even if you do upgrade the code, be sure to have a backwards version of the API that older customers can continue using forever.

Testing Made Simple - Tony Kay

• In order to test something, you often end up writing a second version (generative).
• General principles
  • Don’t clump
  • Don’t involve more than 2 layers at once
  • Name the thing under test and use a “sentence completion” around each test
• Design is more important than most things. If you ever say that your design is really hard to test, then that probably means your design is bad.
• When testing macros in can often be beneficial to use clojure.spec to write conformance. Macros should be tested for the actual code transformations, not for the side effect, that way lies madness.
• https://github.com/untangled-web/untangled-spec

Practical Clojure Profiling in Production - Gregory Sizemore

• YourKit - Great for profiling
• You should profile on production data!
• Reflection is really slow
  • Usually does not matter
  • Type hints can help
• Criterium - benchmarking library for clojure
• Be sure to measure and RECORD every time. You need to be able to “remember” what the performance measures were previously. Don’t rely on yourself remembering how long things took. (NewRelic is recommended)
  • Clj-newrelic - gives you a macro that lets you easily trace functions

Navigating ClojureScript's Fire Swamps - Peter Schuck

• :infer-externs true
• Library cljs/oops
  • Fx oget, oset, ocall, oapply
  • https://github.com/binaryage/cljs-oops
• Soft-access ?, punching !,
• Prefer CLJS library or externs files, when neither exist use clojurescript externs inference or cljs-oops
• Google closure compiler can now compile node modules
  • Npm-deps
• Modules have existed for 2 years! They are really good at splitting. They really suck in terms of all the boilerplate OO code you have to write in order to use them…. Probably still not worth it.
  • Library conwip.modules

Fearless JVM Lambdas - John Chapin

• Austin conference on May 8
• Backend as a Service: Firebase, Cognito, Auth0
• Functions as a Service: AWS Lambda, Microsoft Azure FUnctions, Google Cloud Functions (run in response to events)
• Serverless = Backend as a Service + Frontend as a Service
• Excellent Serverless use cases
  • Latency Tolerance, asynchronous - Data Pipelines
  • Latency tolerant, synchronous - web apps, API’s
  • Glue
• So if you have to use them, then go ahead and use them.

Solving Problems Declaratively - Mark Engelberg

• Dancing Links (exactly one)
  • https://github.com/engelberg/tarantella
  • https://en.wikipedia.org/wiki/Dancing_Links
• Boolean Satisfiability - True/False + logical constraints
  • https://github.com/Engelberg/rolling-stones
  • https://en.wikipedia.org/wiki/Boolean_satisfiability_problem
  • Robot Repair - give the gift of Boolean Satisfiability
• Constraint Programming deals with variables that can take on a finite number of integer values, multi-valued variables + logical and arithmetic constraints
  • https://en.wikipedia.org/wiki/Constraint_programming
  • https://github.com/aengelberg/loco
• http://github.com/engelberg/ycover
• CodeMaster was his previous game
• You should read more Knuth stuff

UNSESSION - Spec - Alex Miller

• Just an question answer pairing on spec.

Full Stack Teleport Testing with Om Next & Datomic - António Monteiro, Mike Kaplinskiy

• At one point they said a parser query can be a mutation? Seems weird.
• Use integration test for correctness in parallel for load testing
• Run test in parallel using multiple databases
• Something about cloning the state of the database so that you can for datomic actions.
  • https://github.com/vvvvalvalval/datomock
• Takeways
  • Datomic is awesome
  • Homoiconicity
  • Om Next reified state transitions (as data!)
  • Immutability (structural sharing)

ClojureScript in Your Pocket - Dom Kiva-Meyer, Lily M. Goh

• ReactNative - Fast Iterations, Familiarity, Cross-platform, Performance, Large Ecosystem
  • Could you only develop ReactNative and just ignore the web
• Expo - Provides additional API’s and allows you to use without needing Mac dependencies. Is somewhat of a “walled garden” of audited features.
  • http://graphql.org/
• Graphql voyager
  • GraphiQl
  • Lacinia - https://github.com/walmartlabs/lacinia
  • Datomic
• Rest - swagger is probably best / graphql tooling so much better
  • http://swagger.io/ - Might be good for gaend
• I still really don’t get how you go about deciding on the schema?

Power to the (Mobile) People: Clojure and GraphQL - Howard Lewis Ship

• GraphQL resultant is the same “shape” as the query
• Some queries are mutations…
• SCHEMA defines the data from graphql’s point of view
• Your job is largely to define functions that handle the different queries

It's Just Data - Bob Calco

• These are good slides, you should look at this more
• A human being with a good visualization can currently understand many things more efficiently than any amount of machine learning.
• Datomic is a great story for understanding the flows of anything through time, including bad data, which can be important for understanding the worlds understanding through time
• Datomic does not have a required field for the same reason it does not have a null value. Irrevocable Trust are kind of cool.
• “Objects and entities are total figments of our imaginations”
• Datomic allows you to simply add attributes as the facts of those models are revealed to you.

Emulators, Immutability, and Time Travel - Angus Fletcher

• NES Emulator
• E.T. almost killed video games
• Nintendo Entertainment System saved it ** PPU is a fancy clock
  • Cycle (vertical) (0-40)
  • Scan Line (Horizontal) (0-261)
• Keeping everything in one place (atom) works really well

Synthesis and Verification for All - Emina Torlak

• A solver-aided tool
• Rusette allows you to implement an interpreter for your language which somehow makes your failing cases pass. I am lost.
• DSL’s are good because
  • They result in less code for humans
  • They have additional constraints that allow tools (like solver-aided languages) that can be exploited.
• The verify logically goes through all paths of the program and ask if any symbolic variables can be given values that cause things to not assert correctly.

I took a 3 week trip to Japan from January 21 from February 12. I didn't have an itinerary, exploring as I saw fit and moving on when the mood struck me. Through the trip, I stayed in Tokyo, Tanabechuo, Kyoto, Osaka, Hiroshima, and Nagasaki.

Purchases

I made several purchases before the trip:

Japan Rail Pass

The Japan Rail Pass (Green) is a Japan Rail (JR) pass that lets you ride most trains in Japan. My most "Japan specific" purchase that I would recommend to anyone. The (Green) designation means that it is business class (effectively), which means better seats and reservations. I bought the 21 day pass for $700.

• You have to buy this pass outside of Japan! It is only available to tourist to Japan. Purchase it before you leave, activate it when you get there.
• This pass lets you ride most intercity trains at no cost. Intracity trains probably are not covered, but they are around $2.
• You can save a little money by NOT using the pass and instead buying the tickets at the station. You can also of course pay less if you take a slower train or a less desirable route. It is up to you to decide whether this is worth your time.
• The pass is the ultimate in flexibility. Just walk up to any JR office and tell them where you want to end up. They book all connecting trains for you then and there. They book you on the next (could be in 5 minutes) train that will get you there as fast as possible. If you have the Green pass they also reserve a seat for you on almost all trains. It is just so easy. You don't need to read local maps, you don't need to plan ahead, just point on your phone where you want to end up.
• I think I basically broke even, but you could go wild on this thing and take a JR trip every day. It is really unlimited.

Google Fi

Google Fi worked out normally. When I landed in Japan it told me that I would pay the standard $10/gig data rate and I believe $0.20/minute. Use your Wifi call if at all possible. Coverage seemed pretty good everywhere. Was able to use it on most of the high speed trains, spotty between Hiroshima and Nagasaki.

Tortuga Outbreaker

The Tortuga Outbreaker 35L was a altogether worthwhile purchase. Most compeling features is the ability to open it fully (not just from the top) like a conventional suitcase. You don't think about it, but not having to pack and unpack your backpack is a real time saver. I also thought the compartment for the laptop with easy access was convenient, as you will be taking that out on a regular basis throughout the day.

Other feature true of the 35L is that it counts as cary on in all airlines (even small planes). It is convenient not having to wait for any lugage after a long day of travel.

hostelworld.com

I used hostelworld.com. Easy to use website. Clean interface. I would book my next hostel 5 days before actually arriving. Never had a single issue. They take I think like 10% as deposit on booking, you then pay the rest upon arrival at the hostel.

Bose QuietComfort 35

I recently bought some Bose QuietComfort 35 headphones for work. This is my best product purchase of 2016. They made air travel more pleasant (shut that baby up!). They made watching the landscape flow by on the trains more moody. They gave me a slight taste of home when I am feeling a bit lonely. Seriously. Great. Headphones.

And now, without further ado, I present... a silly number of pictures:

• Week 1 in Japan
• Week 2 in Japan
• Week 3 in Japan

Google App Engine does not allow for a SCAN like operation on a kind. Although this is a hard limit, it can be worked around with some thought. This is my generic solution.

Considerations

GAE does not allow locking of the database. GAE does not even allow for locking of a kind. Transactions are the only locking mechanism in GAE, and they are limited to the entity groups within the transaction.

The closest GAE has to a scan (a lock of a kind, where every element of that kind is processed) is a GAE Query; which is a filtered iterator. Queries are "best effort" and are not guaranteed to reflect the current state of the database. Still, they are the best you have.

Using a iterator would be a O(N) cost, where N is the number of rows of a GAE kind . GAE is all about scaling dynamically. It would be nice if we could divide up the work of processing every row of a kind.

Solution

I have written a system that lets you (probabilistically) process every entity of a kind. It does not hard guarantee that every entity will be processed, but with a sufficient number of retries it does probabilistically guarantee that they after entity has been processed.

Usage

scan(
  '<kind>', # the Datastore kind that you want to scan
  retries=1, # a non negative integer, default 1
  retry-fx=lambda i: i**2, # i is the retry count, i=0 is always immediate
  max-retries=2, # a non negative integer, default 2
)

The handler-path will be called with the urlsafe() key of every entity of 'kind' <kind>. Specifically /<handler-path>/<urlsafekey> will be called for every <kind> entity. If handler-path returns 2xx, it is considered a success; otherwise it is a failure. Failure will cause the task to be re-enqueued in retry-fx(retry-count) seconds.

Example

Assume there are 3 Dog entities in the database with urlsafekey's of "rottweiler", "poodle", and "dalmation". Then,

from scanner import scan
scan('Dog', "/add/pedigree")

will cause the following endpoints to be called.

• /add/pedigree/rottweiler
• /add/pedigree/poodle
• /add/pedigree/dalmation

This isn't much of a big deal when we are talking about 3 entities. But it is pretty useful when we are talking about thousands or millions of entities. Thanks to this "soft scan" we have the ability to have a function (a endpoint) called on every item of a particular kind in the datastore.

This week I wrote an addition to gaend that automatically causes every ndb.Model entity to automagically be persisted to Elasticsearch.

Why?

Datastore (the database technology that powers Google App Engine) has a number of great features.

1. It scales to like forever
2. With GAE, you can do complex validation on every write
3. Automatically applied security, upgrades, scaling, partitioning, etc.
4. It is pretty simple

Although datastore is perfect as a Key/Value store that allows transactions and limited queries, it does have its limitations.

1. It does not allow SCAN
2. It does not allow JOIN

One effect of these limits is that it can difficult to do things like aggregation on tables. You should use the proper tool for the desired outcome.

• Datastore is great as a key/value store (with transactions).

• Elasticsearch is great at doing aggregation.

Datastore is structured (typed) data. Everything that goes into Elasticsearch must be JSON data. Structured data can be fairly easily transformed into JSON data. You see where I am going with this?

Nuance

This extension to gaend does successfully convert any Datastore data into Elasticsearch. You should look at your table in Datastore and perhaps consider whether you need to submit a custom mapping for Elasticsearch. If unspecified, Elasticsearch will generate a mapping for you based on the submitted JSON document. For tables made up of "primitive" data, the dynamically generated mapping is usually sufficient. Still, know that you can override the dynamic in the event that you have complex, deeply nested, or large types of data.

gaend is a easy way to get Restful endpoints on Google App Engine for Python. It builds endpoints directly from your ndb.Model classes.

Philosophy

This is my first Open Source Python library. It was fun to write and it helped me internalize some of the driving philosophies behind Python. Though there were many learnings, I think the thing I got the most was that Python favors complexity being hidden even at the cost of understanding. In reality, most Python code is NOT simple. However, most python libraries go to great lengths to hide complexity from the user. At some level, I feel that this is a disservice as it risks keeping the user in a permanently "infantile" state. On the other hand, it means that things "just work" and you probably aren't going to cut yourself on a sharp corner.

Contrast this to Clojure. In Clojure, a library should ideally be made of a collection of simple parts. These parts should have no state. It could be bias, but it seems like it is usually easier to understand the internal workings of a random Clojure library vs a random Python library.

The difference though comes down to usage. In Python, a library can often be used with one simple function call. Furthermore, there is usually one blessed library for most problems. Clojure however often requires you to put together "found art" from different libraries. Putting together multiple parts of many libraries to get to a solution. This is the ultimate in flexibility and expressiveness, but it also means that it can be comparatively difficult to get started.

Documentation

Beyond the project itself, I also enjoyed writing the documentation. It can be kind of fun to make believe that you are a user of your software with no context. It can be challenging to try to picture what your own software looks like to a complete stranger. It is actually a rather difficult job and I gained some respect for people who spend most of their lives writing technical documentation.

Your application needs a TOKEN in order to access your Vault server. How do you get the TOKEN to the application server?

Some Background

From here on we will be calling your Vault-server-in-the-cloud simply 'Vault'

Vault is a tool for managing secrets. Vault act as a key/value store where the keys are paths and the values are dictionaries written at those paths. Vault allows for auditing, roles, authentication, and much more.

Vault application access your secrets through a TOKEN. A TOKEN is built from a tuple of (ROLE_ID, SECRET_NONCE). You likely have a single ROLE_ID per application. Your SECRET_NONCE is a cryptographic nonce that is exchanged with the ROLE_ID to get a TOKEN. As a nonce, your SECRET_NONCE may only be exchanged for a TOKEN at most one time.

Once you application has a TOKEN it may then make request to your Vault server. A TOKEN will expire N minutes from its last refresh or initialization. So to keep a TOKEN valid, the application will need to tell Vault at least once every N minutes to refresh (extend) the TOKEN. Your Token will always have a lifetime of (last refresh time) + N. If your server goes down, the TOKEN will soon expire as the application will no longer extend the TOKEN lifetime.

What solutions do we have for secrets?

Don't use Vault; include secrets in your application

We have all done it, it is not a good idea. There definitely should be no secrets in your source code. Secrets in your source code means you cannot share your code without a full audit. Ugh

Environment variables are better. It still requires that you ask questions about what secrets exist at what locations when you are considering your infrastructure. Contrast this to having secrets in one place (Vault). If you are considering Vault then you have likely decided that environment variables are not a good enough solution, so I will not spend any more time attacking them.

Include the Vault TOKEN as part of your deployment

The equivalent of putting ALL your secrets in your application. Anyone who gets access to the TOKEN (through source or environment variables) can access Vault as the application.

Include the Vault (ROLE_ID,SECRET_NONCE) as part of your deployment

This is sound under some strictly ordered conditions. We must be certain that the application is the first to exchange (ROLE_ID, SECRET_NONCE) for the TOKEN. Because SECRET_NONCE is a nonce, we don't need to worry about subsequent request, only the first one wins. However, if Alice is able to get the SECRET_NONCE and exchange it for a TOKEN before our application, Alice would now have access to all of the applications secrets. Dependent order in security does not sound like a good idea.

Pass the Vault (ROLE_ID,SECRET_NONCE) in a application endpoint

After deployment of your app, call an endpoint on it and pass in a (ROLE_ID,SECRET_NONCE). Your application will then exchange your (ROLE_ID,SECRET_NONCE) for a TOKEN. This alleviates the timing concerns of the previous solution while still having zero secrets in code or environment variables. This endpoint can be entirely open. Vault will refuse to provide a TOKEN for a (ROLE_ID, SECRET_NONCE) that it did not issue.

Solution

I have written a demo project that implements a (ROLE_ID,SECRET_NONCE) endpoint. After the very first deployment of the app, it requires that you "light the pilot light" by posting a (ROLE_ID, SECRET_NONCE) that may be use to access Vault. So, the steps are.

1. Deploy the app
2. Make a curl POST request to /vault with your (ROLE_ID, SECRET_NONCE).
3. If you want to update the TOKEN, make another curl POST request with a new secret, expire the old TOKEN.

Instructions are provided in the README.

If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you're spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice. - Donal Knuth

I have an addiction to infrastructure. While programing a solution, I eventually find that I am working on things that do not directly contribute to solving the problem. From a rational/detached/post-fact perspective, I know that this is a huge source of waste. Funny, I never seem to notice this at the time.

The Problem

I do not have a problem with motivation. If anything, I do not value my time as much as I should. Let us leave that for another rant.

I do not have a problem with my programming skill set. It isn't lack of skill that impedes my ability to deliver. Not to say that I have nothing that needs improvement. My mathematics is weak, I never really got SQL, my Git skill are passable but nothing extroardinary, I am average at using the terminal, I type poorly, etc... There are a huge number of things I could learn and improve on, but non of those things stop me from being able to build solutions.

I do spend an embarassing amount of time trying to get things perfect.

I do spend far too much energy trying to deal with eventualities that never emerge.

I do spend mental focus on vanity features that I hope to someday reveal to the world, blinding it with my genius.

Meanwhile, the actual business problem sits untouched.

Observations

Speed is king. I am beginning to believe that this may be true of most things in life. I think the shortest way to get good at something is to become skilled enough to do it quickly. This seems to be true for artist and craftmen, so why shouldn't it be true for programmers?

It seems to me that there are at least 3 elements to speed.

Concurrency

Getting seperate works to come together harmoniously. This is one of the main differences between a good cook and a chef. A good cook often does one thing at a time in a linear fashion. A chef is often doing 3 or 4 things concurrently, connecting them so that they are all completed at the right moment.

Actual context switching in programming is almost certainly a detriment. However, the ability to know the order that things need to be acomplished to minimize downtime is a real skill. Huge numbers of books have been written on software estimation. I am not going to say anything about that. I am going to say that if you have done a task 10 times and it took an average of 15 minutes, then you can probably estimate that the 11th time will take around 15 minutes. This means that you can probably fit this task in a 20 minute lull in your schedule. This makes you faster.

Context Switch (Economy of Motion)

People who have speed aren't always fast. Sometimes they are just more efficient. A good example of this is beginner vs advanced guitarist. Beginner guitarist almost always move significantly more than needed to pick a note. An advanced guitarist does a minimal move. Guess who is going to pick faster (as well as tire more slowly)?

A large part of context switching in programming is familiarizing yourself with the codebase. If you are reusing a codebase that you have previously used, you can dramatically reduce the context switch of familiarizing yourself. This makes you faster.

Actions Per Minute

A measure of the number of task you can perform per unit time. This mostly has to do with repetition. You can perform a task more quickly if you have done a similar task in recent memory. Examples of this are evident.

Programmers may relate to this through the notion of Flow. When you don't have to stop and question how to do something, but can focus simply on what you are doing. This makes you faster as well as less succeptible to distraction.

Solution

Simple. Don't work on the big until you can quickly do the small. That is all I got.

Note: I am picturing a small project being ~10 hours. A medium project being ~50 hours. A large project being 200+ hours.

1. Write down the project name, the difficulty, and the due date given your alloted time. Leave a small space for ✓ or ✗.
2. If you succeed at finishing the project within time, give yourself a ✓ next to the project, otherwise give yourself an ✗. Either way, close the project.
3. Now, ask yourself what the hardest difficulties were. Pick one of these difficulties to fix with infrastructure.
4. Give yourself as much time to write the infrastructure as you spent on the last project. If you fail to get the infrastructure working within time, then stop and go to the next project. This is fine, our goal is to increase speed mostly through your own personal velocity, not through infrastructure.
5. Pick the next Project. If the last 3 consecutive projects with ✓ next to them are of the same difficulty, feel free to move to a higher difficulty project. Loop back to step 1.

Speed

I think it is pretty natural to pick similar projects. Especially if you are trying to get "faster". This should allow you to focus on increasing speed (specifically through Actions Per Minute).

Scratching that Infrastructure Itch

Step 3 & 4 are designed to allow you to address persistent difficulties in code by writting infrastructure. This is very different than my current method of writting infrastructure though as:

1. These are problems that actually came up in the project (no guesswork here).
2. I am limiting how long I can spend on each piece of infrastructure.
3. I am limiting myself to working on only 1 piece of infrastructure.
4. I am stopping infrastructure work if it exceeds its allocated time. Distance lets me become more level headed about whether it is a worthwhile investments. I can choose to resume after another project, but I may decide they it was a false paths. The forced break gives me time for reflection.
5. Note that at most you are spending 50% of your time on infrastructure. I don't know what the right number for that would be, but it is certainly better than I have done on personal projects in the past.

Confidence

Being able to do something quickly and well increases confidence. Computers don't care about confidence, but humans do. If you are reading this, you are likely human. As a human you should look after your mental well being. Being confident in a skill could open a lot of doors...

At the time of this writing, there is exactly one review on Rotten Tomatoes for this film.

A gorgeously shot, tear-stained love letter to female friendship that also provides a fascinating look into contemporary, urban China. - Cary Darling

The above quote was the entirety of my knowledge concerning this film. I think it was "contemporary, urban China" that really capture my interest. I really like movies that might give me some idea of what a foreign country/culture/people are like. Films of this type are harder to find than you would imagine. Very few people capture their own culture in film. Most film captures an idealized version of a culture (for the protagonist), or a lampooned version (for our antagonist).

Best recent good example of this was Hell or High Water, which really capture the manerism/desperation/stagnation of some of West Texas.

I have no real idea whether it succeeds at capturing "contemporary, urban China". As an outsider, it seemed grounded, though a little dramatic.

Review

Soulmate is a Chinese movie that details the lifelong friendship of two women. I liked it. The end.

It isn't the velocity, it is the rate of change. It is the fact that the characters actually move and modify through time. I feel like modern cinema is so concerned with having "strong", "noble", everyman characters that we harden them to the point that they are as mechanical as the plot. I really appreciated the fact that, over years, the two main characters diverge, mirror, and recombine in different ways. That is an ambitious thing to try to make a movie about; especially when you limit yourself to two characters.

The camera work was also fresh. The shots were consistenly set up in interesting ways. People looking across each other in bed. Slow motion scenes to capture moments of youth (implying memory). There were a number of talking head scenes. Even within those, I felt like I was often getting a sense of the room, of the space, within those scenes. I lack the vocabulary to acurately describe what I was seeing, but I could feel it. Somebody really thought hard about how to set up scenes that fit the on screen action.

Reminded me a bit of Our Little Sister, though more maudlin. I thought it was a great movie. It was interesting, I was engaged, and the acting was enchanting.

I like fandom in all forms. There is a good chance that I will attend a conference about anything that you are a fan of. I enjoy being around people who are all interested in the same thing. I am especially attentive when fandom will not yield any status or wealth. This lets me know that these people are actually fans of something. There are no ulterior motives.

Fanimecon is a anime convention "by fans, for fans." Something north of 25,000 people descend on San Jose, California for this thing. I rented a Air BnB with 11 other people. The camaraderie of attending together outweighed the annoyance of waiting for showers. I would recommend attending with a group.

Fanimecon has a game room that is open for the entire length of the conference. This is where I spent a solid 70% of my time. I played euro board games, made acquaintances, and watched competitive gamers. Oh, also socialized and shot the bull. Lots of that.

Cosplay is a big part of these sort of conventions. Some of the costumes were amazing. Myself, I am not that interested in dressing up (though I could be persuaded). I would like to create something to showcase for next years convention. I took real inspiration from Laputa Robot and the Venusaur (below). That is pretty cool.

Cosplay is not all anime/manga based. Around 30% of the cosplays are video game characters. Especially popular this year was Overwatch.

There was also a Manga section where you can relax and read in a less frenetic environment. This is a real boon to people like me, who get agitated and worked up when surrounded by the press of humanity. I took pictures of some of the titles that looked good.

In summary, it was a great time. I met lots of people. I played many games. I even watched a little anime.

I don't own a television. That is no boast, it is a reflection of my poor taste. I am the sort of person that will watch things because they are on. If I owned a television, I would likely spend my days going from Judge Judy to Jerry Springer and back.

I am just as bad at impulse control on the internet. I need a computer to do most... well... everything. For some parts of my day, I need to be working. For others, it is time to sit back, relax, and browse.

I edited my /etc/hosts file so that it redirects sites that are time sinks to localhost. I get a nice This site can’t be reached; no more visiting reddit 3 times an hour!

Sometimes though, I want to actually relax and browse. I then need to edit /etc/hosts again to unblock all those performance killing sites.

Having to edit a file on a daily basis is painful. I can do better.

I am on a Mac, so I use launchd to create two scripts that run at 22:00 and 2:00. The 2:00 script returns my computer to the "focused" state, limiting what sites I can visit. The 22:00 script puts my computer in a "relaxed" state, allowing me to browse until bedtime. Both of these files should be in your /Library/LaunchDaemons directory. The advantage of putting these files in /Library/LaunchDaemons as opposed to ~/Library/LaunchAgents is that they are run whenever the computer is on, rather than just when you are logged in. Also, they will run upon wake, even if the trigger time occured while the computer was asleep.

Daemonic Agents - Code example that includes /etc.hosts files that I find useful

This week I joined a study session / book Club for Haskell. The Book we are covering is Haskell Programming

History

I joined rather late so had to do 6 chapters in one week. Now that I am caught up I can do a leisurely 1 chapter a week. Everyone reads and performs the exercises in each Chapter on their own. We have a weekly meetup where we can talk about any difficulties with the exercises, get a head start on the next chapter, or just socialize.

Why?

Mostly to just reaffirm my belief that what I have is already the best. I really enjoy Clojure. It is my favorite language to work within. The only aspects I sometimes question are related to testing and correctness. I think the unit/functional/generative testing story is great in Clojure. Test are neccesesities in Clojure in order to generate correct code; is this a universal truth?

I have this fantasy though that maybe, just maybe, if I understood type systems well enough I could write code that does not need test at all. Code that is so bulletproof that the fact that it compiles is all the proof we need to know that it works. I suspect that this is a fantasy. Unfortunately, I don't think I can convince myself of this unless I can actually use a strongly typed language and demonstrate this to myself.

Impressions

Less than impressed.

I get that they wanted a language that is similar to how mathematical functions are written. This results in some rather complex and ridiculous rules for using operators. Ok. Fine.

I also think the fact that I can choose either where or let just makes the code harder to understand as I have to move my eye all over the place to grasp what is going on.

The type system is of course a painful thing to bear with, but is probably also the aspect of Clojure that will pay the most dividends. I find it challenging, at the same time I am excited as I begin to understand how it all works.

Conclusion

We march on! Haskell is one of those things that I have said I would learn for years. Here is an opportunity to do so. Doesen't come up all that often. I don't care if I am not getting any immediate value out of it. This is a bucket list item and I am completing it.

Hatsune Miku is a popular vocaloid performer. I heard about vocaloid singing (and Hatsune Miku) a few years ago. I have never listened to vocaloid music previous to this concert. I decided to attend one as I find the concept intriguing. This concert was held at the Warfield.

Tent Revival for Nerds?

I have not attended many conerts. I am trying to remember the last one I attended; probably more than a decade ago now. I believe the last was either "A Perfect Circle", "Nine Inch Nails", or KMFDM (can't remember which was last). Whenever I am at a concert, I am always struck by how similar concert experiences are to religious ones. People reaching towards the heavens, putting their hands up, sometimes clossing their eyes and letting the feeling of the moment capture them.

I am not a religious person. Similarily, at a concert, I have trouble "letting go" and getting worked up with the crowd. I try to, but I often fail. Anyway, enough about me.

This concert was similar to any other concerts in that the people in it were experiencing a "spiritual" communion with the singers and with each other. The fact that the singers were entirely artificial, with voices digitally stitched together from (1 or more, not sure) human singers seems to make no real difference. The love of the singers, music, and genre was real.

Glitch Mob Sound

Having said that, the music really takes some getting use to. For the first 10 minutes I was thinking little more than "Jeesus, this shit is atonal and the voice is just horendous". I was putting on a good face because I wanted the people I was with to have a good time (and they did) but I was internally thinking "Gaaaawwwdd, this is awful". The sound grew on me after a half hour or so. I don't think it is music I am going to listen to in a non concert setting, but it does grow in the context of a live showing.

On a Meta Level

I think the whole idea of worshiping Idols that have never existed is probably the next stage of things. Already, most comercial bands are 90% fabrication. Really just companies with a product looking to maximize market share and future earnings. When you see a Katy Perry concert, you are mostly seeing the decisions and work of the huge number of people who go towards fabricating that image. Yes, there is an actual 'Katy Perry', but she is just the human form of the Platonic ideal of a Katy Perry. That ideal is a fiction created by a huge number of people. This is the next logical step; why bother even maintaining the fiction of a real personf?

Did I enjoy myself?

Actually. Yes. Immensely so.

At first I was feeling kind of nervous and disoriented about the fact that the singer was artificial. However, you start to tune in on the fact that everyone around you is ecstatic to be here. There are no posers here, nobody looking to make a scene. No pretentious ass from your local theater district. These are people who are embracing something that is, to put it bluntly, rather (totally) dorky. And they love it! They are doing it because they love the music, the character, the sound, whatever. It is refreshing to see people enjoying something so much.

It may have been the glow stick "orchestration" (I got pretty good at it). It may have been the screaming crowds that just couldn't get enough of one act after another. It may just have been the chance to see people I know who are usually very reserved belting their hearts and souls to songs that I can't even follow (all Japanese). I don't know, doesen't really matter. I enjoyed myself; would go again.

Odd film. It kind of reminded me a bit of "The Fountain", though this one is a bit more subdued.

Hah, maybe that creepy Mayan (I forgot) looking guy who keeps saying "Death is the road to Awe" is Karamakate... Just kidding. Didn't even seem like the philosophies were all that compatible.

Recording & Encryption

I had vague knowledge about "rubber plantations" and just how poorly native pupulations were treated during those times. Awfull. Things like that really reaffirm my commitment to the idea of a monitoring society that makes easy use of recording technology + encryption. Recording so that you can have an "unbiased" (<- really loose use of the term there) view of something and encryption so that you can share said unbiased evidence anonymously. I can't promise a better world. But it seems to me that a great number of evil is done because not enought people of power are aware of them. Recording + Encryption does not fix the world, but it at least gives a voice to and causes note of it for others.

Pride

As I watched the film, I was reminded how little "pride" I have in my own culture/race/whatever. Karamakate loves his culture, he wanted his people to continue, barring that, he wanted the knowledge of his culture to endure. Above that, he views the jungle and his gods as being a sort entity that cared for him and whom he cared for. Not that I am in any way ashamed of my culture, it is just rather nebulous; not something I give a great deal of thought. I an American by birth. I am North Western European (British, French, German) by lineage? I have no religion. I have no membership in any permanent organization. Hell, the closest thing I have to an identity is probably that of "programmer". I know that other programmers are going to have similar mindsets to my own. This is a group, I guess it is even a culture, but it is not something I take inherent pride and strength from. I largely see myself as an independent agent exerting smal force upon this world. I don't really see myself connected to anything, I only see the influence of myself upon it. Hard to describe, but Karamakae seemed to see himself as something within a whole, where he and the whole were in a sort of balanced (I wanted to use the word 'homeostatic' here, but that is too fancy) eternal state. I don't have anything like that; anything I want to preserve above myself. It is sometimes an empty feeling, at the same time, it means that I am beholden and owe fealty to nothing.

A random list of file conventions that make me happy. Although these do not apply to every situation, they are true enough that you should at least consider them when creating an output file format.

If it is a list of things, use CSV

Don't make up your own format. Please. Just use CSV (note: does not have to be commas, any unused delimiter will work, or use quoting characters within your CSV if you have to).

Don't use slashes in the filename

This hurts. If you include slashes in the filename, it makes is look like the file is actually a directory listing. Furthermore, modern storage system like gcs and s3 don't really have directories (they are really just a bunch of objects in a bucket). However, they will logically present these objects as if they are made up of directories if you include slashes in the names. This is odd because when you download individual files they will download as files. When you view them the files within the s3 viewer they will appear as directories. when you (r)sync the buckets the files they will appear as directories... It just gets messy. Just don't use directories. Use a flat bucket of files.

Include everything of a homogenous type within a single directory/bucket

I think buckets should basically be treated like typed arrays/vectors/list. You should only have a single type of thing within them. Don't mix multiple different types of things as you are then forcing someone using said bucket to filter on what they need. Really, if you have multiple types of things, use multiple buckets.

Make it so the names of files are automatically sorted in some usefull way

Let me provide an example.

1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/89/0/rtp.pcap.wav.ctm
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/9/0/rtp.pcap.wav.ctm
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/90/0/rtp.pcap.wav.ctm

9 should not logically be between 89 and 90. However, most directory listing operations will list things according to logical text order. Ugly. I think it it would be much better to sufficiently pad such a file so that 9 does not show up here, something like

1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/00008/0/rtp.pcap.wav.ctm
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/00009/0/rtp.pcap.wav.ctm
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/00010/0/rtp.pcap.wav.ctm
...
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/00089/0/rtp.pcap.wav.ctm
1946304563_66189429@216.221.154.11-66.228.112.5:27832/callee/00090/0/rtp.pcap.wav.ctm

Of course, this involves either:

1. Just picking some arbitrary padding that will always be sufficient (famous last words)
2. Extending the padding on all previous files whenever you add another digit. So, for instance, when you are on element 10 you go through all previously written files and change 0 to 00, 1 to 01, 2 to 02, etc.

Let me just say how much I appreciate you doing this for me. You have no idea how anoying it is to have to always itertools.groupby(sorted(os.listdir(path), key=key_fx1), key=key_fx2) whenever your filenames does not logically sort. Tiresome.

On Bovine women, plebian taste, and Renoires non Impressionist work.

A bit of a jump

I was interested to learn that Renoir actually had a fairly major "shift" in drawing style. Many people know Pierre-Auguste Renoir as an Impressionist painter. Few realize that he seems to have tired of this style in his 50's and switched away from Impressionist style art and towards (mostly female) nudes. Although his art from before his 50's (1890's) is commended, the work after is often considered appalling by critics for various reasons.

Bovine women

One of the critics in the film commented that Renoir drew "bovine" "empty headed" women. I am going to ignore the "empty headed" part, as hell, I don't know. A lot of classical paintings seem to be of people with queer expressions on their faces, I guess I haven't thought about it enough.

Bovine. This struck me as somewhat odd. Foremost, he liked big girls, what is the big deal? Secondly, it is an interesting use of language. In modern times, calling anyone bovine, no matter how fat they are, would be an incredibly rude thing to say. This critic clearly did not like Renoir's work. However, it appears that it is ok to use terms that you would personally find in bad taste if you are using those terms to describe the behavior of someone whom you are casting in a bad light. Interesting. The critic would probably never directly call someone bovine. He would however use the term when describing the output of someone he dislikes. It is a subtle verbal trick, but in doing so it allows one to use a distastfull but powerful word without attributing it to yourself. I thought that was clever.

Plebeian taste

Hell, I liked most of what I saw. The art makes you feel good. It is relaxing. It reminds you how attractive the female form is and makes you want to go lay in a sunny field. What is not to like about that?

A recording of all that I saw and hear at Clojure West 2016 in Seattle Washington.

Nathan Marz Specter

Github

1. Navigation is independent abstraction
2. Navigation dissasociated from the desired transformation
3. Provides type defined defaults for navigation to nil
4. Assuming you can write a generic enough function

This library lets you navigate and modify deeply nested and deeply repetitive data structures. Fundamently, it raises the barrier before you need to start putting things in a database by making it possible to just reason about data when the data is very deep or very broad. It makes the actionable code look like a DSL in its conciseness; without actually being a DSL. Having said that, it is bringing in a rather large "mini DSL" in order to allow for the data manipulation.

Donevan Dolby

This was a walkthrough of a 16,000 line of code Clojure App. Lessons were learned, libraries were used, and discarded. Code is being used on the Boeing 737 MAX; which is evidently a new plane with a focus on efficiency.

Claire Alvis

https://github.com/SparkFund/spec-tacular

An attempt at putting some sort of stronger typing on entities in Datomic. Graded various aspects of her attempt. Did not entirely follow due to lack of familiarity with Typed systems in particular.

1. Adds types in to Datomic by having a spectacular shema -> becomes datomic schema
2. Verification of code
3. Documentation is specified by the spec

Michał Marczyk

This was just a deep dive into exactly what a record is and when it is appropriate to use one. A record is just a collection of fields, possibly of different data types, typicially with a fixed number and sequences.

Records also takes types into account (.equals and .hashCode)

Big exception is that = and .equals are NOT the same when comparing Record to a map even with the same value.

dissoc a key from a Record returns a Map

It can be a bit surprising to see this. Unfortunately, they often still look like records. In fact, this is one of the reason record? exist.

Reasons you might use records

1. Clear data semantics (you are naming things) (clara-rules)
2. computed properties (midpoint of a circle, IShape, Circle, Rectangle)) (core.async)
3. polymorphic 'actions' (Stuart Sierra's component)
4. A clean simple way to create a named class implementating a interface
5. (defrecord ForJava [...] :load-ns true ...)

Mario Aquino

Showcased an API for interaction to the Alexa Skill system from the Clojure API. It is called Boomhauer, from the king of the hill character. GITHUB

1. Interactions can be single exchange or can be stateful
2. Alexa has two concepts Intent and slots

• Intent - What you want to do GETHOROSCOPE
• Slots - The "arguments" passed to a

3. Uterances - How you might express the things you are interested in eg.
4. There is a local development that you can do
5. Mobile could provide a great way to make Alexa a real success.
6. There are libraries that generate all the uterances you want from a grammar.
7. There is a Certification process. Got to get approved like Mac Application.

This might be good for the sales reps. I mean, they are already wearing a headset, it would be cool if they could do things like take voice notes and make calls and all that stuff without even having to touch the screen.

Michael Drogalis

1. "Peerless" in the sense that peers open direct line of communication between each other
2. Zookeeper for keeping track of the entry's that have been added

I have no use for this, thought it was neat. It is basically a system for doing distributed computation in a peerless system.

Jack Dubie

Hiearchy of testing - [Decoupled, End to End, Lightweight, Simple]

1. Unit Test
2. API Test
3. UI Test (Selenium) - Just test what is supposed to happen

How do you test that something was done without actually tying to a particular db?

Refresh the page after performing an action, you should have the same state as you had when you finished. Without tying to a particular thing you have proven that it has been persisted.

Stuart Sierra

1. Removing a method definition does not remove it from the ns
2. Macros if changed will not cause things using the macro to be re-evaluated
3. Protocols don't reload because Java Classes are Immutable

Assumptions

1. 1 file = 1 ns
2. No circular dependencies
3. No loading code outside of ns
4. No fully qualified ns in variables, you must require everything

Dealing with Application state

• RELOADED
• COMPONENT
• TOOLS.NAMESPACE

You can have multiple systems that are seperate "systems" in development and production

When you need development, just merge all the systems into one map

tldr; Source Code != Program

Tony Kay

UNTANGLED

Honestly, it looks rather complex to me. I do not convinved these new models based on Falcor are actually superior.

Bozhidar (Cider)

l - can be used to show a list of the locals
o - takes you out of the current debugger
s - stacktraces
which-key package -
http://cider.readthedocs.org/en/latest/
there is a really good story in terms of being able to trace recursive functions
enlighten mode basically enables lightable in emacs
cider-scratchpad is usual for prototyping ideas
C-c C-e for normal evaluation
C-j to print out the result
cider-inspect lets you look at complex data
cider-docs - to see docs
cider-grimoir - to see examples
cider-apropo - match all symbols related to the word you name
Projectile installed allows you to toggle between implementation and test

Questions

Can you instrument the debugger from API?

Yes.

Can you debug Clojurescript?

No.

Zach Tellman (Clojure Applied)

• The extra verbosity of using standard functions rather than ...
• Always be composing - https://www.youtube.com/watch?v=3oQTSP4FngY
• Dynamic Variables for Datastore & TaskQueue
• with-redefs might be better? Not sure.
• Most of the time you want to be verbose rather than cryptic.
• It is better to duplicate code than to have the wrong abstraction

Arachne

Some modular web framework that is going to normalize how people do web programming.

Mikaela Patella

• Web Developer -> Distributed Systems Engineers
• The clients and the servers are all segments of a distributed system.
• The "server" is an abstraction over the entire backend, a consistent snapshot of an uncertain distributed system
• Big idea is that "Frontend" and "Backend" are relics of the past (We are really just distributed systems.)

Think of as games....

What if we thought of it less as "request/response" (webdev) programming and more as a game? Where we have to do local level rendering of a segment of the entire world.

Priyatam Mudivarti

• Invalidation - How do you inform other users that the current information is no longer valid
• Eviction - How long do I exist?
• Naming - What is th ename of the cache
• {cache-name}-{uid}-{cache-version}-{logical-timestamp}
• In order to properly invalidate things, include a timestamp

So, for good caching, also include a timestamp as part of the UUID, this way you don't have to revert data, instead you can just increment the timestamp and this will mean the cache is no longer used. Thi smight actually apply to Elasticsearch for my own uses.

Alex Kehayias (Chocolatier)

Problem with conventional game engines is

1. you intermingle state and behavior
2. Not declarative
3. No Interactive development

Entity Component system

• Dungeon Siege & Unit Game Engine
• Basically, it is a map of components (interfaces) that it supports
• Devcards - This guy had a good experience with devcards.

Performance

• Variadic function scannot be optimized (use multiple arities instead)
• apply has a real cost (it can not be optimized)
• keyword-equal? is much faster
• defmulti is slow, detype, defrecord, defprotocol is fast
• AVOID INTERMEDIATE COLLECTIONS
• If you now it is a boolean, skip test use ^boolean

Some great Resources at the end

Matthias Felleisen

• All types think types while they create code
• In some programs, we can check programs at compilation time
• He is against Type Inference.
• Hindley-Milner type inference (Ml, Haskell)

Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.

Personal

Good conference. Met a number of people that we can bring in to work if we need their skills. Learned about new technologies in the Clojure community. Specifically, I think the Alexa system might be something we should look into for Reps. Specter might be good for keeping our system as Data and not needing to put everything in a database. The Data science team might like that; of course, it would involve learning clojure. I think that would be a win win for everyone. :]

Awful. Just awful. This is a bad Harlequine Romance novel wrapped in the thinnest veneer of Sci-Fi. Let me transcribe some of this crap for you...

> "You're beautiful."
>
> "I bet you say that to every naked woman you wake up in bed with."
>
> "No", he said in all seriousness. Before he realized what he'd done. Then he turned bright red. "I mean", he was too late!
>
> [Name Redacted to avoid spoilers], "Got you!" she giggled, holding him down. Then she subsided on top of him.

What. The. Fuck. This is some of the more event driven "dialogue" in the book. When our Mary-Sue-Wonder-Bread-Social-Warrior isn't spending most of her day conducting internal monologues about how she feeeeels at every moment, we get the treat of listening to this level of dialogue between herself and her compatriots.

Enough about the dialogue, let me get into the parts of the book I really hated.

HERE BE SPOILERS



Understand the culture you are in.
I really dislike it when literature treats other races/cultures/peoples by only characterizing them by the mores and beliefs that we disagree with. It is just so one sided, so arrogant; really gets my heckles up.

In this case, I was just flummoxed how our heroine is able to just waltz in and start instituting change without anyone being able to touch her because she can outfox them due to her free market understanding of how the world really works.

Question your environment
I could not buy that an investigative journalist expressed no real interest in figuring out how the locket/worldwalking worked. It just blew my mind. She didn't even ask if anyone had ever done any research on it! Come on!

Hell, I would have given her a pass if she had at least bothered to figure out the parameters, the limits of this thing. Nope, the second she figured out that it could work, we were done. After discovery, she never again bothers to test the limits of the system, to question why there is a mass limit, why do you get headaches, is their conservation of momentum? Nothing. It was just so... I just couldn't believe this. It is a good thing she dropped med school, she would have been a terrible doctor; doubt she could diagnose a cold.

Nobody cares about your feeeeeelings
I was so sick of hearing how she "wasn't going to allow them to ...." or "Let them try to get me to ...". As a verbal tick it gets old fast. The forever monologues as she connects her painfully slow reasoning made my eyes water.

The book just ends
Literally nothing is resolved at the end. It just ends. Asshole. In terms of interest, it is just dull for about 85%, then you have 10% of rising interest, and then you have a 5% ending where you realize nothing is going to complete. Wow, that made me angry.



END OF SPOILERS

I really have a hard time believing this is the same guy who wrote "Accelerando", "Rule 34", hell even "Neptunes Brood". Amazingly Bad.

Big idea here is that the Universe is a spectrum of fast to slow zones. In fast zones we have access to "higher" levels of computation, in slower zones the exact same machinery will become buggy or nonfunctional. It was never specified exactly what computations you can do in higher zones that you cannot do in lower zones. The important side effects of this though are that higher zone allows self aware AI and allows ships to jump between the stars. Lower zone have no self aware machinery and requires that you actually travel between the stars at only a small multiple of lightspeed. It is strongly implied that humanity/Earth may have come from a lower zone. That is very rare, as most "higher zone" aliens actually accend by being contacted by a more evolved aliens. Humanity may have actually manually made the voyage to higher zone from lower zone without outside intervention, though most aliens would regard that as unlikely.

In the highest zones (the Transcend), there are AI's that are basically Gods. Within their zones they have almost unlimited power. Although ultimately constrained to the higher zones, the projection of their power extends to lower zones. Fortunately, most AI's in the Transcend quickly loose interest in the going on's of life in the lower zones. Metaphors fail in terms of applying lower zone reasoning or emotion to a God, but you might say that it is considered perverse by most AI's to be at all interested in the life of lower real beings. They view us the same way human beings might view insects. Some human beings actually have pity and take interest in insects, but the majority simply leave them alone as they have no real interest in them one way or another. I liked that part of the story. I like the idea of an AI that basically isn't even evil, it just does not care because you are beneath it's notice.

I liked the idea of the Zones. I liked the idea of all intelligence (from virus all the way to Artificial Intelligence) basically being some form of computation. The lowest zone, known as the "Unthinking Depths", does not even allow biological species to achieve sentience. I liked the way Intelligence was just quantified as being something that is only limited by physics. It was a neat idea.

The other big idea I liked was the Tines. The Tines are an alien species where each "person" is made up of 4 to 8 members. Each of these members looks something like a greyhound (at least in my mind). Each member is a distinct organism, but is usually not capable of any advanced thought. Members communicate at the speed of sound by constantly talking to each other. Once a group of 4+ members forms and acclimatizes they will become sentient. In doing so, the new "person" is a combination of the personality, characteristics, and skills of all its constituent members. Once a member joins, they almost never leave a group; it is not a confederacy of members, it is a person that came to be because of it's constituent members. I really liked this idea. I thought it was a very creative way of thinking about some new type of intelligence. A intelligence that came to be in a different way and for a different reason than human intelligence. It makes me think of split brain patients, how they are one person, but in some (very unscientific) sense of the word have two brains. I really liked how the speed of sound was a limiting factor to their group intelligence. I recollected some of the amazing feats of plasticity that the human brain seems capable of. What would the result be of linking seperate human beings brain-to-brain with radio? It also made me think a good deal about how similar thinking about an experience is to actually experiencing it. Imagine, in a Tines group (person), one member has his nuzzle in a flower, breathing in the pollen. He is communicating the sensation to the other members in his group. For all intents and purposes, they all simultaneously have their nose in that flower. It is a really interesting idea. I think of intelligence as being singular, but must it be? What would it mean to have trully parallel intelligence? Where each "node" is broadcasting and receiving everything it experiences and thinks to every other member in the group? It sounds wonderfull.

Saw "Sicario" this evening. It was a pretty decent flick. Remined me a bit of the submarine movie I watched a while ago called "Black Sea"; left you with the same tense feeling in your neck. I guess, I guess I enjoy movies like that. I like movies where things are just building up slowly, where tensions are being accrued. I think I also respond to the idea that this is everyday life for the people in these films. I wonder how I would (or if I could) handle those sort of situations.

I guess the take home message of the film is that the only way to deal with lawless entities like the Cartels would be to meet them with violence. The state, as a system of laws, is not really capable of inflicting the sort of violence neccessary to check them. The state therefore looks to outside contractors to do their violence for them. Is this acceptable? Is the result actually better? Is the premise even sound?

I did enjoy the contrast between the clean cut FBI agents and the result driven CIA agents. I can't honestly decide who is right in these sort of situations. Of course, I don't know whether the movies portrayal of each side correctly personifies reality. It was still an interesting character study.

Wasn't bad. As a time saver, I would almost recommend reading it in reverse.

This book contains three parts. The first part is 3/4 of the book or so "The Atrocity Archive". The second part is a short story in the same world with the same character within "Atrocity" called "The concrete Jungle". The last part is a reflection by Charles Stross about books of this nature.

So the last part of this books waxes philosophical and otherwise about the nature of a book like this in the world. It is probably what you should read first. If what is described within sounds interesting to you, then read from the beginning. If it sounds awful, then read something else.

If, however, you aren't sure, then I would recommend reading the second story "The Concrete Jungle" first. Did you like that? Then you will probably like the first (and much longer story) "The Atrocity Archive". There, I made efficient use of your time.

Personally, this really wasn't my thing. If felt too much like Star Trek dilithium-crystal-deus-ex-machina kind of mumbo jumbo to me. To be fair, I don't enjoy horror (maybe it is a lack of imagination), so I may have just been skimming when I was supposed to be enthralled.