Category: Research

Posted on

2 Sides of the Same Coin: Residual Blocks and Gradient Boosting

No fun backstory for this post. These things are just cool and important in machine learning. Turns out they are very algorithmically similar. So I’ll explain how 🙂

Residual Blocks Review:

A residual block in a neural network is defined by the presence of skip layers, as seen in the id(Hk-1) connection in figure (1). These skip layers represent the addition of a layer i-k so some layer i . These blocks reduce the effect of vanishing gradients, and enable the development of much deep neural networks. This architecture enabled Microsoft to win the 2015 ImageNet challenge, and residual blocks have become a staple of deep neural networks.

Image result for resnet block
Resnet Block | Credit: Andrew Ng

Continue reading “2 Sides of the Same Coin: Residual Blocks and Gradient Boosting”

Posted on

Know Thyself: And Know Thine Uncertainty

So in my recent readings of various ML media, from blog posts to published papers, I’ve just started to notice a trend. I’m growing more certain that the ML community is ignoring uncertainty, which is certainly not a good practice and it renders much their results quite uncertain.

In this post, I wanted to just go over a quick and easy method to use inverse probability to estimate the uncertainty in your model’s test-accuracy. Continue reading “Know Thyself: And Know Thine Uncertainty”

Posted on

Bad Optimizers, Black Boxes, and Why Neural Networks sometimes seem just Backwards-ass Lucky

Woah! TensorFlow! Neural Networks! Convolutionation Recurrent Deep Learned Blockchain Etherium Network. Where’s the line start??

How much can I spend?

Okay, maybe the last one isn’t actually a thing (for all I know). But there is currently a lot of hype and excitement around deep learning, and for good reason. Neural networks have provided a number of improvements in performance, and specific fields such as computer vision, speech recognition, and machine translation have been genuinely revolutionized by deep learning.

With that said, this will be Part 1 of the Grind my Gears series, where I will be talking about Deep Learning issues that just really grind my gears. This will be a less mathematic post than usual, but I will link to resources to dive in deeper if you are interested. With that said, let us begin:

Continue reading “Bad Optimizers, Black Boxes, and Why Neural Networks sometimes seem just Backwards-ass Lucky”

Posted on

BFGS, Optimization, and Burritos

Hey ya’ll. Hope your summer is going well! In the season of peaches and watermelon, it’s easy to take things for granted. No one has experienced this more than the optimization algorithms used in, well, just about every single machine learning and computational problem.

I thought, for this post, I would dive into one of the classics: the Broyden Fletcher Goldfarb Shanno algorithm, also known as BFGS, named after these ballers right here.

Continue reading “BFGS, Optimization, and Burritos”

Posted on

Canetis: Forced Alignment Post-Game

So for those of you who don’t know, I work as a student researcher at SAIL, a signal analysis lab at USC. To read more about my work with them, check out my earlier post in which I describe the various ongoing project at the lab.

My current main project, along with my friend Nihar Sheth whose github you can find here, has been the development of a forced alignment algorithm that works for long and/or noisy Continue reading “Canetis: Forced Alignment Post-Game”

Posted on

Forced Alignment: Pre-Game

Hope ya’ll’s day is going well! In my last post, I talk about the lab, SAIL, that I’m working at and an overview of the ongoing projects at the lab.  Today, I’m going to talk the specifics of my current role within the Forensic Interview Project.

For those of ya’ll who didn’t get the chance to read my last post, the problem that SAIL is currently trying to solve is the high level of

Continue reading “Forced Alignment: Pre-Game”