Recent blog posts by Andrej Karpathy at Medium.com and Pete Warden at PeteWarden.com have caused a paradigm shift in the way I think about neural nets. Instead of thinking of them as powerful machine learning tools, the authors instead suggest that we should think of neural nets, and in particular, convolution deep nets, as ‘self-writing programs.’ Hence the term, “Software 2.0.”
It turns out that a large portion of real-world problems have the property that it is significantly easier to collect the data than to explicitly write the program. A large portion of programmers of tomorrow do not maintain complex software repositories, write intricate programs, or analyze their running times. They collect, clean, manipulate, label, analyze and visualize data that feeds neural networks. — Andrej Karpathy, Medium.com
I found this to be a dramatic reversal in my thinking about these techniques, but it opens up a deeper understanding and is much more intuitive. The fact is that combinations of artificial neurons can be used to model any logical operation. Therefore you can conceptualize training a neural net as searching programming space for an optimal program that behaves in the way you specify. You provide the inputs and desired outputs, and the model searches for the optimal program.
This stands in contrast to the “Software 1.0” paradigm where the programmer uses her skill and experience to conceptualize the right combination of specific instructions to produce the desired behavior. While it seems certain that Software 1.0 and 2.0 will co-exist for a long time, this new way of understanding deep learning is crucial and exciting, in my opinion.
Recently, I’ve been doing some research into Hailey-Hailey Disease (HHD). HHD is an autosomal dominant genetic disorder that leads to severe dermatosis. The disease causing variants are located in the ATP2C1 gene, which is a magnesium-dependent, calcium transporting ATPase.
There are unfortunately few treatment options for HHD. Many treatment options have been tried, from corticosteroids to tacrolimus. There are very few HHD patients, and therefore no large scale clinical trials of therapies for this disease.
I came across a paper that shows that a novel approach, low-dose naltrexone (LDN), may be an effective and low-cost therapy for treating HHD. What is more remarkable, however, is the fact that LDN has already been used with success to treat many diseases like fibromyalgia, Crohn’s disease, and HIV.
Here is the complete list of diseases that LDN has been used to treat with some success according to some case reports and small-scale clinical trials:
Adenoid cystic tongue carcinoma
Chronic eczema and pruritis
How is LDN effective across so many seemingly unrelated diseases? I can’t really answer that question. We do know that naltrexone is an opioid receptor inhibitor that is used in the treatment of alcohol and opioid abuse at higher doses. At low dose, the mechanism of action is less clear, but some studies suggest increases in beta endorphins and suppression of cytokines using LDN.
As of now, LDN remains an “off-label” use of naltrexone and in the realm of internet anecdotes until more rigorous studies can be completed. Regardless, it is an exciting development in the potential treatment of rare diseases, like HHD.
Here are five different ways to gain an introduction to the topic of CNNs. Each approach is geared toward a different style of learning:
Visualize them in real time with your own inputs (this is amazing!)
Watch a lecture by the “godfather” of neural nets, Geoff Hinton.
Take a top-ranked online course on Deep Learning.
Learn the math behind them.