On advice from my uncle I'm continuing to fallback on task difficulty with RNNs.
Unc's tips: - Swirch to generation task - Try residuals - Go deeper - Add projections - No dropout?
Let's recreate Karpathy's classic post and train a language model on tiny-shakespeare. We can get the entire dataset which is a text …
read moreToday I'm going to recreate the pipeline shown in lesson 10 of the fast.ai course. We'll go through what's needed on the high-level, using pretrained models for everything. The pipeline is fed in a text prompt and it produces an image. A prompt means we need a tokenizer to …
read moreI'm pivoting the RNN summarization code to an easier example - Machine translation. Easier in the sense of the dataset, which consists of much shorter en-de sentence pairs compared to the summarization task. I have some suspicion that the there is a bug or something in my code, so today, after …
read moreToday we're gonna dip our fingers into the first generative NLP task - text summarization. We're gonna use the CNN/Daily Mail dataset as done in this paper. Let's get to it.
I started by doing all the preprocessing of the files myself, but then found a the dataset …
read moreread more"It’s better to do something simple which is real. It’s something you can build on because you know what you’re doing. Whereas, if you try to approximate something very advanced and you don’t know what you’re doing, you can’t build on it." - Bill Evans …
If you're like me, you don't remember trig identities. If you're like me, you prefer rederiving stuff yourself. It's a perk of math that we replace memorization with derivation. Still, it takes time, and distracts you from what you're actually doing. Also, deriving trig identities isn't so easy without some …
read moreI expect another couple of days before I can submit a pull request. Today after some horrendous debugging I finally got some tangible progress. status, log, add-all and commit all seem to be working, perhaps even init (but still inconclusive). Still need to add push and pull. Then I need …
read moreI reached a milestone today in my project to integrate git into the mobile version of logseq. Finally the git functions like add, status, init were working. I also realized that all the work and time that I put in so far was to writh this file system adapter that …
read moreIt's been a couple more days now since my last post on the subject. I spent another 10 hours or so, and haven't gotten much further. Mostly I'm stuck with file system problems. In my last post I said I would use filesystem abstraction already present in …
read moreOriginally I intended to write about a physics problem I did today, but then this turned into more of a journal thing, so feel free to skip it.
After another week, a recurring thought that is troubling me is my productivity. I question myself again and …
read more