Trying to make an LLM my 'personal assistant'

I have written about local LLMs several times now. News summarization, the boundaries of what they are good and bad at, and mechanisms for automatic collection.
This time, I want to go a step further and talk about using them to 'have an LLM help with daily tasks'.
I am running GPT-OSS-20B (quantized) in LM Studio on my EVO-X2.

Not 'having my job taken by AI'

When the topic of AI comes up, it tends to lean toward 'jobs being taken away', but
that is not my experience.

At least at this stage,
I think AI is closer to a 'tool that helps with work' rather than an 'entity that does the work for you'.
Just as a kitchen knife doesn't cook a meal for you, an LLM depends on the person using it.

However,
having good tools definitely changes your work efficiency.

Beyond summarization and translation

In a previous article, I wrote about using LLMs for news summarization and translation.
That is convenient in its own way, but as I continue to use it, I feel it is a waste to only use it for that.

Since GPT-OSS-20B has 20B parameters,
it can handle more than just simple summarization.

I will write about a few specific examples.

First, brainstorming text.
For this Note article, instead of just starting to write, I sometimes ask the LLM, 'I want to write about this theme, what kind of structure would be good?'
(This article is like that too)

I don't use the suggestions that come out exactly as they are, but it helps organize the vague ideas in my head.
There are quite a few times when I think, 'I wouldn't have thought of that angle,' and it is a good brainstorming partner that never complains.

Next, drafting code.
In a previous article, I wrote, 'Local is fine for short functions and decoding error messages, but generating code over 100 lines falls apart.'
That is correct, and I don't dump large code tasks on it.

But for things like Python script skeletons or configuration file templates,
it is perfect for delegating tasks that 'I can write, but are a pain to look up'.
Between writing from scratch and modifying a draft, the latter was overwhelmingly faster.

Also, making research more efficient.
When researching something, I first ask the LLM to get an overview.
Then, I search and verify as needed.

The LLM's answers aren't always accurate, but
'what keywords should I use to search?'
it is enough to get a sense of that.
Especially in fields with many technical terms,
just getting an initial foothold saves a lot of time.

The meaning of running it locally

For these uses, you can do the same thing with ChatGPT.

So why local? There are a few reasons.

One is ease of use.
There is no process of opening a browser and logging in.
If you keep LM Studio running on your EVO-X2, you can use it anytime.
It is a trivial thing, but this feeling of 'being able to ask immediately' is important, and you can resolve small questions on the spot without letting them pile up.

Another is privacy.
There are times when I don't want to send work content or personal ideas to the cloud.
If it is local, everything is completed within my own PC, so
no matter what I ask, it doesn't go outside.

And, cost.
Even if I use it dozens of times every day, there is no additional charge.
With cloud APIs, you are charged based on usage, but local only costs electricity.

The feeling of an 'assistant'

As I continue to use it,
it starts to feel more like an 'assistant' than a 'tool'.
It is not a perfect secretary, but it feels like there is someone I can ask to do chores.

Sometimes it says things that are off the mark, and
I can't entrust it with difficult things.

But,
simple research, drafts of text, generating a bit of code, etc.,
I delegate things one after another that 'take 5 minutes if I do it myself, but come out in 1 minute if I ask'.

That accumulation changes the amount of work I do in a day.

It is easy to use because it is not omnipotent

It is a strange way to put it, but
because local LLMs are not omnipotent, they are easy to use.

When sending long instructions to ChatGPT,
I sometimes unconsciously feel pressure, thinking, 'I'm paying for this, so I need to get good results.' With local, that doesn't exist.

I can just think, 'If it fails, I'll just ask again.'

This low psychological hurdle
leads to using it a lot, which ultimately increases total efficiency.

Next time, I plan to write an introduction to my desk environment. Please follow me!