- Valk3_This one is pretty cool that i saw some time ago, not a game though, but a 3D portfolio: https://www.jessezhou.com/. The author even goes into how he made it: https://jesse-zhou.medium.com/jesses-ramen-case-study-77bae7....
- I've been thinking about the problem of what to do if the answer to a question is very different to the question itself in embedding space. The KB method sounds interesting and not something I thought about, you sort work on the "document side" I guess. I've also heard of HYDE, the works on the query side, you generate hypothetical answers instead to the user query and look for documents that are similar to the answer, if I've understood it correctly.
- What I'm wondering about is, if you have lots of dependencies, like in the hundreds or thousands, idk how many npm packages usually can have for the average web dev project, how do you even audit all of that manually? Sounds pretty infeasible? This is not to say we should not worry about it, I'm just genuinely curious what do you do in this situation? One could say well don't get that many dependencies to begin with, but the reality of web dev projects nowadays for instance, is that you get alot of dependencies that are hard to check manually for insecurities.
- How do you become good at this? Is it simply practising doing UI/UX interfaces over and over and over and over until you get it? Do you have some specific advice on how to get "there" faster? I recently stumbled across this Refactoring UI book[1], heard it's sort of popular, know if it's any good?
- Looks very well executed, congratulations! I'm curious what tech stack did you use to develop the site?
- Not sure what your exact task is, but I have a similar goal as well. Haven't had time to try alot of different models or ideas yet because got busy with other stuff, but have you tried this: https://youtu.be/dQ-4LASopoM?si=e33FQd5f4fYr4J5L&t=299
where you stitch two images together, one is the working image (the one you want to modify), and the other one is the reference image, you then instruct the model what to do. I'm guessing this approach is as brittle as the other attempts you've tried so far, but I thought this seemed like an interesting approach.
- Thanks for the answer! I think you are right, I've also heard of HYDE (Hypothetical answer generation), that makes an LLM encode a guess as the answer into the query, which may also improve the results.
- Sorry for my lack of knowledge, but I've been wondering what if you ask a question to the RAG, where the answer to the question is not close in embedding space to the embedded question? Will that not limit the quality of the result? Or how does a RAG handle that? I guess maybe the multi-turn convo you mentioned helps in this regard?
The way I see RAG is it's basically some sort of semantic search, where the query needs to be similar to whatever you are searching for in the embedding space order to get good results.
- At one hand you get insane productivity boost, something that could take maybe days, weeks or months to do now you can do in significantly shorter amount of time, but how much are you learning if you are at a junior level and not consciously being careful about how you use it, feels like it can be dangerous without a critical mindset, where you eventually rely too much on it that you can't survive without it. Or maybe this is ok? Perhaps the way of programming in the future should be like this, since we have this technology now, why not use it?
Like there's a mindset where you just want to get the job done, ok cool just let the llm do it for me (and it's not perfect atm), and ill stitch everything together fix small stuff that it gets wrong etc, saves alot of time and sure I might learn something in the process as well. And then the other way of working is the traditional way, you google, look up on stackoverflow, read documentations, you sit down try to find out what you need and understand the problem, code a solution iteratively and eventually you get it right and you get a learning experience out of it. Downside is this can take 100 years, at the very least much longer than using an llm in general. And you could argue that if you prompt the llm in a certain way, it would be equivalent to doing all of this but in a faster way, without taking away from you learning.
For seniors it might be another story, it's like they have the critical thinking, experience and creativity already, through years of training, so they don't loose as much compared to a junior. It will be closer for them to treat this as a smarter tool than google.
Personally, I look at it like you now have a smarter tool, a very different one as well, if you use it wisely you can definitely do better than traditional googling and stackoverflow. It will depend on what you are after, and you should be able to adapt to that need. If you just want the job done, then who cares, let the llm do it, if you want to learn you can prompt it in certain way to achieve that, so it shouldn't be a problem. But this sort of way of working requires a conscious effort on how you are using it and an awareness of what downsides there could be if you choose to work with the llm in a certain way to be able to change the way you interact with the llm. In reality I think most people don't go through the hoops of "limiting" the llm so that you can get a better learning experience. But also, what is a better learning experience? Perhaps you could argue that being able to see the solution, or a draft of it, can be a way of speeding up learning experience, because you have a quicker starting point to build upon a solution. I dunno. My only gripe with using LLM, is that deep thinking and creativity can take a dip, you know back in the day when you stumbled upon a really difficult problem, and you had to sit down with it for hours, days, weeks, months until you could solve that. I feel like there are some steps there that are important to internalize, that LLM nowdays makes you skip. What also would be so interesting to me is to compare a senior that got their training prior to LLM, and then compare them to a senior now that gets their training in the new era of programming with AI, and see what kinds of differences one might find I would guess that the senior prior to LLM era, would be way better at coding by hand in general, but critical thinking and creativity, given that they both are good seniors, maybe shouldn't be too different honestly but it just depends on how that other senior, who are used to working with LLMs, interacts with them.
Also I don't like how LLM sometimes can influence your approach to solving something, like perhaps you would have thought about a better way or different way of solving a problem if you didn't first ask the LLM. I think this could be true to a higher degree for juniors than seniors due to gap in experience when you are senior, you sort of have seen alot of things already, so you are aware of alot of ways to solve something, whereas for a junior that "capability" is more limited than a senior.
- This is not in my area of expertise, so I apologize if this question is bad, but can blender do more in terms of creating 3D models than OpenSCAD? If so, wouldn't it mean that if there was a blender equivalent of this, then the capabilities for 3D modelling would be even greater? I have to guess that it's harder to achieve something like this in Blender than in OpenSCAD?
- Thanks!
- This might be a vague question, but what kind of intuition or knowledge do you need to work with these kind of things, say if you want to make your own model? Is it just having experience with image generation and trying to incorporate relevant inputs that you would expect in a 3D world, like the control information you added for instance?
- Regarding the math in ML, what I would love to see (links if you have any) is a nuanced take on the matter, showing examples from both sides. Like in good faith discussing what contributions one can make with and without a strong math background in the ML world.
edit: On the math side I've encountered one that seemed unique, as I haven't seen anything like this elsewhere: https://irregular-rhomboid.github.io/2022/12/07/applied-math.... However, this only points out courses that he enrolled in his math education that he thinks is relevant to ML, each course is given a very short description and or motivation as to the usefulness it has to ML.
I like this concluding remarks:
Through my curriculum, I learned about a broad variety of subjects that provide useful ideas and intuitions when applied to ML. Arguably the most valuable thing I got out of it is a rough map of mathematics that I can use to navigate and learn more advanced topics on my own.
Having already been exposed to these ideas, I wasn’t confused when I encountered them in ML papers. Rather, I could leverage them to get intuition about the ML part.
Strictly speaking, the only math that is actually needed for ML is real analysis, linear algebra, probability and optimization. And even there, your mileage may vary. Everything else is helpful, because it provides additional language and intuition. But if you’re trying to tackle hard problems like alignment or actually getting a grasp on what large neural nets actually do, you need all the intuition you can get. If you’re already confused about the simple cases, you have no hope of deconfusing the complex ones.
- I've only skimmed through both of them, so I might be entirely incorrect here, but isn't the essential approach a bit different for both? The MIT one emphasis not to view matrices as tables of entries, but instead as holistic mathematical objects. So when they perform the derivatives, they try to avoid the "element-wise" approach of differentiation, while the one by Parr et Howard seems to do the "element-wise" approach, although with some shortcuts.
- I wonder what kind of contributions can you make with a strong math background versus someone with just undergrad math background (engineer)? I know it's a vague question and it's not so cut and dry, but I've lately been thinking about theory vs practise, and feel a bit ambivalent towards theory (even though I started with theory at first and loved it) and also a bit lost, mostly due to the steep learning curve, i.e. having to go beyond undergrad math (CS student with undergrad math background). I guess it depends on what you want to do in your career and what problems you are working on, but what changed my view on theory was looking at other people with little math background or with only undergrad math background at most, that still were productive in creating useful applications and or producing research papers in DL, which showed to me that what is more important is having a strong analytical mind, being a good engineer and being pragmatic. With those qualities it feels like you can go top-down approach when trying to fill in gaps in your knowledge, which I guess is possible because DL is such an empirical field at the moment.
So to me it feels like the "going beyond undergrad math" formally is more if you want to be able to tackle the theoretical problems of DL, in which case you need all the help you can get from theory (perhaps not just math, but even physics and other fields might help as well to view a problem through more than one lens). IMO, it's like casting a wide net, where the more you know the bigger the net is and hope that something sticks. Going the math education route is a safe way to expand this net.
- I'm curious, are you able to use it to read text at night when it's dark? Because that's probably the only reason I would use it, but at that point it would be better to get an e-reader or something, I guess.
- Awesome, looking forward to it!
- I'm kind of late to this post, but really awesome initiative and well executed project! Thank you for bringing this to people!
I tested it a bit and it seems pretty decent, although for some really niche theoretical questions it wasn't successful in retrieving the answers I wanted even if alot of the results were really good in other aspects. It could simply be because the answer is not available anywhere in hackernews.
I'm wondering if someone were to build a similar project but for other sites, what would your advice be? For instance what technical difficulties did you stumble on that you think would be good to be aware of?
Thanks in advance and once again congratulations on the project!
- Great work! I got a question though, what's the processing time for longer videos? Like lecture videos that usually are somewhere around 1h-2h? Can it handle longer vids reasonably fast?