We recently began a Visiting Scholar
where researchers and professors can spend some time at our offices and
learn about HTM in depth. Designed to promote collaboration,
participants play an active part in our research meetings while
continuing their normal research. To give you a better understanding of
this new program, I interviewed our first visiting researcher, Mirko
Klukas, and asked him about his time spent at Numenta.
Mirko Klukas, Ph.D.
Hi Mirko, can you introduce yourself and at a high level, your area of research and expertise?
My name is Mirko Klukas and I am a post-doctoral researcher at the
Institute of Science and Technology Austria. The
core of my current research is a combination of mathematics and computer
science, including topological data analysis and computational topology.
My background lies in pure mathematics in the field of geometric and
differential topology. Believe me, it sounds worse than it actually is. 🙂
How did you learn about Numenta?
In 2013 I was on a vacation in Sweden, in a typical Swedish country
house with a pretty unique set-up: no running water, but WIFI. I read
the first Numenta white paper on that vacation – basically between
chopping wood and keeping the fire burning – and kept thinking about the
approach ever since. How the paper actually did end up in my hands in
the first place? I have no clue.
At my current position I enjoy a lot of freedom when it comes to my
research, and I devoted a significant amount of time to think about the
approach from the paper and related questions. Last year I reached out
to Subutai to see if there is any interest in my ideas.
What was a typical day (or week) like for you during your time here?
It felt very familiar, the atmosphere was very calm and relaxed, and a
typical day didn’t differ too much from my usual research days with two
exceptions. I could discuss things with the team, and I tried to do that
as often as possible. This could happen in smaller one-on-one meetings
using a whiteboard, or by giving a talk on my ideas to more members of
the team. In addition I had the chance to attend basically all regular
meetings, non-research meetings included. Personally I found that to be
very interesting as it gave me an idea about the operations side and the
other facets of the company.
What did you hope to get out of it?
The part of my research that is related to Numenta’s technology is only
very loosely connected to my mathematical research and it was great to
have a counterpart to bounce ideas back and forth. It was great to talk
and pitch ideas to people that speak the same language, and I am pretty
confident and excited that down the line this collaboration will result
in a publication.
What was most memorable or valuable to you?
Numenta finds a nice balance between its research mission and being
product driven. The atmosphere and people at Numenta reflect that. In my
experience the two ends of this spectrum usually dance to a very
different beat, and it was great to experience a research environment
outside of academia, where I felt comfortable and could imagine myself
working. I felt really welcome, which I value a lot. Being able to
present my ideas to Jeff and the rest of team was a unique experience
Was there anything you learned that you could not have learned without being here?
It was inspiring to be at the cutting edge of Numenta’s current
research, and it opened up new perspectives on previous ideas. Looking
over someone’s shoulders when they are in the process of solving a
problem is something that shouldn’t be undervalued. Talking to Jeff,
Subutai, and the rest of the team, and getting their perspectives is a
great asset. In particular, I appreciated getting a glimpse on where the
individual priorities and emphases lie.
How will your time at Numenta shape your future work?
As mentioned earlier, I hope my time at Numenta wasn’t an isolated
event, and my visit was just the starting point for many future
discussions to come. It also encouraged me to pursue the direction
towards machine intelligence research even more.
Can you describe for more technical readers the area of your specific research?
I will spare you a detailed explanation of my mathematical research. 🙂
If someone is interested in that, all my pure math papers can be found
arXiv – some
keywords are: low-dimensional topology, contact and symplectic
topology, symplectic cobordisms, open book decompositions, Engel
My current research focus in the machine learning and intelligence realm
revolves around two main directions.
How can you generalize sequence learning approaches for discrete
alphabets, e.g. variable markov order models like context tree
weighting or prediction by partial match, to non-discrete input
spaces, namely constant weight codes, or sparse distributed
representations respectively? This can be done, for instance, for the
incremental parsing algorithm, the main ingredient in the classical
1978 compression algorithm after Lempel and Ziv. The result is a
recurrent neural network structure very similar to Numenta’s sequence
memory. This construction could serve as a reference point to highlight
or contrast certain aspects of Numenta’s sequence memory, e.g. it could
help to proof theoretical bounds on its prediction performance.
The second focus revolves around binary sparse coding, and Numenta’s
spatial pooling algorithm in particular. The algorithm is closely
related to the witness complex, a construction of simplicial complexes
well-known in computational topology – you find a small note about the
connection of the spatial pooler and the witness complex on my blog:
Maps, Functions and
Generally I try to establish a meaningful role of entropy within binary
sparse coding and contrast it to other binary coding approaches. This is
loosely related to expressivity of neural networks, and plays an
interesting part in the objective of the spatial pooler. The objective
balances a good reconstruction error and maximal mean individual
entropy. The interplay of these two quantities is something I want to understand better as well.
If you’re interested in applying for the Visiting Scholar Program, [click here](/company/careers-and-team/careers/visiting-scholar-program/) to apply.