We recently began a Visiting Scholar Program, where researchers and professors can spend some time at our offices and learn about HTM in depth. Designed to promote collaboration, participants play an active part in our research meetings while continuing their normal research. To give you a better understanding of this new program, I interviewed our first visiting researcher, Mirko Klukas, and asked him about his time spent at Numenta.
Mirko Klukas, Ph.D.
Hi Mirko, can you introduce yourself and at a high level, your area of research and expertise?
My name is Mirko Klukas and I am a post-doctoral researcher at the Institute of Science and Technology Austria. The core of my current research is a combination of mathematics and computer science, including topological data analysis and computational topology. My background lies in pure mathematics in the field of geometric and differential topology. Believe me, it sounds worse than it actually is. 🙂
How did you learn about Numenta?
In 2013 I was on a vacation in Sweden, in a typical Swedish country house with a pretty unique set-up: no running water, but WIFI. I read the first Numenta white paper on that vacation – basically between chopping wood and keeping the fire burning – and kept thinking about the approach ever since. How the paper actually did end up in my hands in the first place? I have no clue.
At my current position I enjoy a lot of freedom when it comes to my research, and I devoted a significant amount of time to think about the approach from the paper and related questions. Last year I reached out to Subutai to see if there is any interest in my ideas.
What was a typical day (or week) like for you during your time here?
It felt very familiar, the atmosphere was very calm and relaxed, and a typical day didn’t differ too much from my usual research days with two exceptions. I could discuss things with the team, and I tried to do that as often as possible. This could happen in smaller one-on-one meetings using a whiteboard, or by giving a talk on my ideas to more members of the team. In addition I had the chance to attend basically all regular meetings, non-research meetings included. Personally I found that to be very interesting as it gave me an idea about the operations side and the other facets of the company.
What did you hope to get out of it?
The part of my research that is related to Numenta’s technology is only very loosely connected to my mathematical research and it was great to have a counterpart to bounce ideas back and forth. It was great to talk and pitch ideas to people that speak the same language, and I am pretty confident and excited that down the line this collaboration will result in a publication.
What was most memorable or valuable to you?
Numenta finds a nice balance between its research mission and being product driven. The atmosphere and people at Numenta reflect that. In my experience the two ends of this spectrum usually dance to a very different beat, and it was great to experience a research environment outside of academia, where I felt comfortable and could imagine myself working. I felt really welcome, which I value a lot. Being able to
present my ideas to Jeff and the rest of team was a unique experience as well.
Was there anything you learned that you could not have learned without being here?
It was inspiring to be at the cutting edge of Numenta’s current research, and it opened up new perspectives on previous ideas. Looking over someone’s shoulders when they are in the process of solving a problem is something that shouldn’t be undervalued. Talking to Jeff, Subutai, and the rest of the team, and getting their perspectives is a great asset. In particular, I appreciated getting a glimpse on where the individual priorities and emphases lie.
How will your time at Numenta shape your future work?
As mentioned earlier, I hope my time at Numenta wasn’t an isolated event, and my visit was just the starting point for many future discussions to come. It also encouraged me to pursue the direction
towards machine intelligence research even more.
Can you describe for more technical readers the area of your specific research?
I will spare you a detailed explanation of my mathematical research. 🙂 If someone is interested in that, all my pure math papers can be found online on arXiv – some keywords are: low-dimensional topology, contact and symplectic topology, symplectic cobordisms, open book decompositions, Engel structures.
My current research focus in the machine learning and intelligence realm revolves around two main directions.
How can you generalize sequence learning approaches for discrete alphabets, e.g. variable markov order models like context tree weighting or prediction by partial match, to non-discrete input spaces, namely constant weight codes, or sparse distributed representations respectively? This can be done, for instance, for the incremental parsing algorithm, the main ingredient in the classical 1978 compression algorithm after Lempel and Ziv. The result is a recurrent neural network structure very similar to Numenta’s sequence memory. This construction could serve as a reference point to highlight or contrast certain aspects of Numenta’s sequence memory, e.g. it could help to proof theoretical bounds on its prediction performance.
The second focus revolves around binary sparse coding, and Numenta’s spatial pooling algorithm in particular. The algorithm is closely related to the witness complex, a construction of simplicial complexes
well-known in computational topology – you find a small note about the connection of the spatial pooler and the witness complex on my blog: Maps, Functions and Arrows. Generally I try to establish a meaningful role of entropy within binary sparse coding and contrast it to other binary coding approaches. This is loosely related to expressivity of neural networks, and plays an interesting part in the objective of the spatial pooler. The objective balances a good reconstruction error and maximal mean individual entropy. The interplay of these two quantities is something I want to understand better as well.
If you’re interested in applying for the Visiting Scholar Program, click here to apply.