this post was submitted on 22 Dec 2024
521 points (96.1% liked)
Technology
60103 readers
2025 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I know I'm an enthusiast, but can I just say I'm excited about NotebookLLM? I think it will be great for documenting application development. Having a shared notebook that knows the environment and configuration and architecture and standards for an application and can answer specific questions about it could be really useful.
"AI Notepad" is really underselling it. I'm trying to load up massive Markdown documents to feed into NotebookLLM to try it out. I don't know if it'll work as well as I'm hoping because it takes time to put together enough information to be worthwhile in a format the AI can easily digest. But I'm hopeful.
That's not to take away from your point: the average person probably has little use for this, and wouldn't want to put in the effort to make it worthwhile. But spending way too much time obsessing about nerd things is my calling.
From a nerdy perspective, LLMs are actually very cool. The problem is that they're grotesquely inefficient. That means that, practically speaking, whatever cool use you come up with for them has to work in one of two ways; either a user runs it themselves, typically very slowly or on a pretty powerful computer, or it runs as a cloud service, in which case that cloud service has to figure out how to be profitable.
Right now we're not being exposed to the true cost of these models. Everyone is in the "give it out cheap / free to get people hooked" stage. Once the bill comes due, very few of these projects will be cool enough to justify their costs.
Like, would you pay $50/month for NotebookLM? However good it is, I'm guessing it's probably not that good. Maybe it is. Maybe that's a reasonable price to you. It's probably not a reasonable price to enough people to sustain serious development on it.
That's the problem. LLMs are cool, but mostly in a "Hey this is kind of neat" way. They do things that are useful, but not essential, but they do so at an operating cost that only works for things that are essential. You can't run them on fun money, but you can't make a convincing case for selling them at serious money.
Totally agree. It comes down to how often is this thing efficient for me if I pay the true cost. At work, yes it would save over $50/mo if it works well. At home it would be difficult to justify that cost, but I'd also use it less so the cost could be lower. I currently pay $50/mo between ChatGPT and NovelAI (and the latter doen't operate at a loss) so it's worth a bit to me just to nerd out over it. It certainly doesn't save me money except in the sense that it's time and money I don't spend on some other endeavor.
My old video card is painfully slow for local LLM, but I dream of spending for a big card that runs closer to cloud speeds even if the quality is lower, for easier tasks.
Nvidia's new motto: "An A100 at every home"
I’ll pay a bit more for the next model of my phone that promises on device ai, or actually already did. We’ll see if that turns into something useful.
So far the bits and pieces I’ve played with are not generative ai, but natural language processing and inferencing. The improved features definitely make my phone a more useful piece of hardware, but not revolutionary
Being able to summarize and answer questions about a specific corpus of text was a use case I was excited for even knowing that LLMs can't really answer general questions or logically reason.
But if Google search summaries are any indication they can't even do that. And I'm not just talking about the screenshots people post, this is my own experience with it.
Maybe if you could run the LLM in an entirely different way such that you could enter a question and then it tells you which part of the source text statistically correlates the most with the words you typed; instead of trying to generate new text. That way in a worse case scenario it just points you to a part of the source text that's irrelevant instead of giving you answers that are subtly wrong or misleading.
Even then I'm not sure the huge computational requirements make it worth it over ctrl-f or a slightly more sophisticated search algorithm.
Multiple times now, I've seen people post AI summaries of articles on Lemmy which miss out really, really important points.
Isn’t this what the best search engines were doing before the AI summaries?
The main problem now is the proliferation of AI “sources” that are really just keyword stuffed junk websites that take over the first page of search results. And that’s apparently a difficult or unprofitable problem for the search algorithms to solve.
That's what Google was trying to do, yeah, but IMO they weren't doing a very good job of it (really old Google search was good if you knew how to structure your queries, but then they tried to make it so you could ask plain English questions instead of having to think about what keywords you were using and that ruined it IMO). And you also weren't able to run it against your own documents.
LLMs on the other hand are so good at statistical correlation that they're able to pass the Turing test. They know what words mean in context (in as much they "know" anything) instead of just matching keywords and a short list of synonyms. So there's reason to believe that if you were able to see which parts of the source text the LLM considered to be the most similar to a query that could be pretty good.
There is also the possibility of running one locally to search your own notes and documents. But like I said I'm not sure I want to max out my GPU to do a document search.
Even the success case is a failure. I’ve had several instances where Google returned a nice step by step how to answer a user’s questions, correctly, but I can’t forward the link and trust they’ll see the same thing
Well an example of something I think it could solve would be: "I'm trying to set this application up to run locally. I'm getting this error message. Here's my configuration files. What is not set up correctly, or if that's not clear, what steps can I take to provide more helpful information?"
ChatGPT is always okay at that as long as you have everything set up according to the most common scenarios, but it tells you a lot of things that don't apply or are wrong in the specific case. I would like to get answers that are informed by our specific setup instructions, security policies, design standards, etc. I don't want to have to repeat "this is a Java spring boot application running on GCP integrating with redis on docker.... blah blah blah".
I can't say whether it's worth it yet, but I'm hopeful. I might do the same with ChatGPT and custom GPTs, but since I use my personal account for that, it's on very shaky ground to upload company files to something like that, and I couldn't share with the team anyway. It's great to ask questions that don't require specific knowledge, but I think I'd be violating company policy to upload anything.
We are encouraged to use NotebookLLM, however.
You're using the wrong tool.
Hell, notepad is the wrong tool for every use case, it exists in case you've broken things so thoroughly on windows that you need to edit a file to fix it. It's the text editor of last resort, a dumb simple file editor always there when you need it.
Adding any feature (except possibly a hex editor) makes it worse at its only job.
... I don't use Notepad. For anything. Hell, I don't even use Windows.
Not sure where the wires got crossed here.
Yes as others said, the op mentioned notepad and you said notebookllm.
I thought you were talking about notepad and it's new ai features.
I had no idea notepad + AI was a thing. It sounds farcical, so I assumed wrongly it was a reference to NotebookLLM. My mistake. I shouldn't have assumed OP was just being dismissive.
Then either you replied with your first post to the wrong post or you misread "windows putting AI into notepad" as notebookLLM? Because if not there is nothing obvious connecting your post to the parent
I don't think anyone is putting AI into Notepad. It reads to me like a response to NotebookLLM but maybe I was wrong.
I did at least explain what my vision is and why I wanted it which.... doesn't sound anything like Notepad, I think.
Well, you think wrong: https://blogs.windows.com/windows-insider/2024/11/06/new-ai-experiences-for-paint-and-notepad-begin-rolling-out-to-windows-insiders/
Might be, but the person you responded to wrote about windows putting AI into notepad, so everyone assumed you were responding to that and not writing about something that was not even mentioned
I stand corrected. Thank you. I hadn't heard about that. Notepad has always been no frills, and I can't see integrating AI with that over just using AI, but they are and it seems silly, I agree.