alankarmisra
Joined 67 karma
- alankarmisra parentThey’d presumably do worse. LLMs have no intrinsic sense of programming logic. They are merely pattern matching against a large training set. If you invent a new language that doesn’t have sufficient training examples for a variety of coding tasks, and is syntactically very different from all the existing languages, the LLMs wouldn’t have enough training data and would do very badly.
- Same. I use Apple Notes. I have a few notes pinned (regular work, creative work, self-education, travel, chores). I write tasks. Break them up into small tasks with indents. Pick a task from the pool and execute. "Regular Work" tasks get priority. But if I'm not feeling it, I move to the other ones. Once I finish a task, I delete it/replace it with next steps. Nothing fancy. No formatting except for indentation. Been crushing it.
But I will add, there is no right way to do things in life in general. Experiment, and do what works for you.
- Genuinely curious; while I understand why we would want a language to be open-source (there's plenty of good reasons), do you have anecdotes where the open-sourceness helped you solve a problem?
- It's like the secret beaches in every south-east asian nook and crany. They're so secret there's signs pointing to them every where and they are overrun with tourists.
- I see the value in showcasing that LLMs can run locally on laptops — it’s an important milestone, especially given how difficult that was before smaller models became viable.
That said, for something like this, I’d probably get more out of simply finding an existing implementation on github or the like and downloading that.
When it comes to specialized and narrow domains like Space Invaders, the training set is likely to be extremely small and the model's vector space will have limited room to generalize. You'll get code that is more or less identical to the original source and you also have to wait for it to 'type' the code and the value add seems very low. I would rather ask it to point me to known Space Invaders implementations in language X on github (or search there).
Note that ChatGPT gets very nervous if I put this into GPT to clean up the grammar. It wants very badly for me to stress that LLMs don't memorize and overfitting is very unlikely (I believe neither).
- My thinking is threaded. I maintain lists (in a simple txt file and more recently, in Notes on the Mac) and add the tasks to it. Subtasks go into an indent. I have different notes for regular work/pet project/blog/learning/travel. priority-must-do-now/daily chores is separate one. Every morning I open my priority/daily chores stuff and try and wind that up. And then I just scuttle around the other lists and do whatever my brain tells me I can. I find that some days I do more from the blog notes and some days more from the regular work notes. The notes serve as goals for my brain and it invents/discoveres solutions in no particular order. This makes me more productive because I can switch when I'm bored (which to me is an indication that my brain needs more time to find solutions in this space). And if nothing is hitting the right note, I'll take a nap or read or watch a show for a bit or go for a long walk or hike - anything that's not in the to-do just to give myself the creative space. I find that giving myself problems to solve, and allowing my subconcious brain to invent solutions for it while I do other things actually works quite well for me and allows me to make steady progress.
- I would argue that framework isn't the winning component, the people are. A lot of people can say similar things for framework <<X>> and they'd be right given their own experience but I think they give themseleves too little credit. I've written my own frameworks for smaller projects because I didn't want the needless learning curve / cruft of a generalized framework and I can tell you they saved me so much time / effort / cost over the long term and they were tuned to my development style. All built with python and some off-the-shelf libraries. I'm not saying there's no place for generalized frameworks. They do help streamline development efforts and set a standardized model for building out your app/service which would be great with a team of people all tuned in to a specific framework but I would argue that in a single developer environment it doesn't matter much what you use, so long as you have fun with it. Productivity analyses are at best personal opinions in a forum like this.
Edit: basic grammar.
- I would guard against "arguing from the extremes". I would think "on average" compact is more helpful. There are definitely situations where compactness can lead to obfuscation but where the line is depends on the literacy and astuteness of the reader in the specific subject as already pointed out by another comment. There are ways to be obtuse even in the other direction where written prose can be made sufficiently complicated to describe even the simplest things.
- In your specific example, time of day, weather (foggy, sunny, over-cast) along with images of cars with different colors, models, makes, from different angles will all be training parameters to begin with so the neural net can do this on its own without needing specific symbolic processing apriori or downstream. Training data input into neural nets is usually sanitized and transformed to some extent but whether this sanitization / preprocessing requires symbolic programming depends on the use case. For example, with the car example, you preprocess car images to color them differently, hide random sections of it, clip it in different ways so only partial sections are showing, turn them upside down, introduce fake fog, darken, lighten, add people, signs, fire, etc and use each of these images for training so that the neural net can recognize cars under different situations (even after accidents where they are upside down and on fire). Eventually the neural net will recognize a car in most circumstances without symbolic programming/intervention.
So when would you use symbolic programming? To generate quality data for the neural network. For example, maybe the neural net reports it read the speed limit to be 1000 km/h on a sign because of someone's shenanigans. A symbolic programming aid which knows potential legal limits will flag this data as potentially corrupt and pass it back to the network as such allowing the neural network to take more sensible decisions.
- I'm wondering if this is a limitation though. If it can be learnt from training data, would it not be part of the neural network training data? I imagine we use Scallop to bridge the gap where we can't readily learn certain rules based on available data or perhaps we would prefer to enforce certain rules?
- I read the paper on Lobster a little bit. Scallop does its reasoning on the CPU - whereas Lobster is an attempt to move that reasoning logic to the GPU. That way the entire neurosymbolic pipeline stays on the GPU and the whole thing runs much faster.
- It's a combination of neural networks and symbolic reasoning. You can use a neurosymbolic approach by combining deep learning and logical reasoning:
A neural network (PyTorch) detects objects and actions in the image, recognizing "Jim" and "eating a burger" with a confidence score.
A symbolic reasoning system (Scallop) takes this detection along with past data (e.g., "Jim ate burgers 5 times last month") and applies logical rules like:
The system combines the image-based probability with past symbolic facts to infer: "Jim likely likes burgers" (e.g., 85% confidence).likes(X, Food) :- frequently_eats(X, Food). frequently_eats(Jim, burgers) if Jim ate burgers > 3 times recently.This allows for both visual perception and logical inference in decision-making.
- This. I'm trying to set up a personal developer blog and I have a very specific set of requirements. Tried several static blogging frameworks. Apart from the software bloat, I found myself spending a gratituous amount of time trying to customize pieces to my needs. Finally got sick of it, landed up writing a python script and after a few days and < 200 lines of code - I have a working prototype that fits my current needs. I will be doing more of the wheel re-inventing for other projects I have in mind. I strongly agree with your observation that adapting generic frameworks to specific needs probably have longer learning curves than just building a new wheel.
- This paper suggests that LLMs can be trained to handle multi-stage questioning by automatically optimizing prompts using feedback-based methods, improving their ability to process complex, multi-step interactions.
- Same. I was writing my own language compiler with MLIR/C++ and GPT was ok-ish to dive into the space initially but ran out of steam pretty quickly and the recommendations were so off at one point (invented MLIR features, invented libraries, incorrect understanding of the framework, etc) that I had to go back to the drawing board, RTFM, and basically do everything I would have done without GPT to begin with. I've seen similar issues in other problem domains as well just like you. It doesn't surprise me though.