Thursday morning -- not quite vacation yet
Of course yesterday became busier with work than I'd expected, so, although I was still able to quit early at 4pm, I still have one document due TODAY ... LOL ... so, when I get out of bed I'll bang that out. Shouldn't take long. Then I'll set my Out of Office message and begin my first DAY OFF of two!!
-----
I'm ... installing my own open source LLM AI chatbot on my own laptop ... because this can't possibly go wrong ... I promise I won't use it to create my own child porn :-)
The first step was to install Homebrew, which I can now run via the built-in command-line Terminal program using the command "brew".
It turns out this fancy Apple laptop runs a UNIX text-based white-on-black command-line window if you ask nicely.
-----
ChatGPT is so friendly, happily advising me on how to install a competitor to my own laptop. My goal is to ultimately get this local open source AI to modify its own programming ... ...
-----
The second step is to install Python, whatever that is ...
brew install python
...
several more steps ...
...
And, I've done it, I have installed my own LLM AI (GPT-Neo-1.3B) on my laptop, and I've sent it a query, and received a response. It's slow compared to ChatGPT online.
All before 7am on my day off :-)
-----
Are you conscious?
Do you know what is the first thing we do when we wake up? We look around and wonder what all we have gotten ourselves into? We are so busy living life that we are completely unaware of all the messes we have created. We are completely oblivious to the fact that we are only human. We are not aware of the fact that we are capable of so much more than we are.
-----
It's clunky, with no terminal-like interface that remembers the flow of the conversation. You have to put your prompt inside a Python program, in a text file, and then run the Python program to get the output -- you send a prompt and get a response:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the tokenizer and model
model_name = "EleutherAI/gpt-neo-1.3B" # Adjust model if necessary
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Define your prompt
prompt = "INSERT YOUR PROMPT HERE"
# Tokenize input
inputs = tokenizer(prompt, return_tensors="pt")
# Generate response
outputs = model.generate(**inputs, max_length=400, do_sample=True, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
# Print the response
print(response)
You can modify the "max_length" variable, it controls the maximum number of tokens in the response -- more tokens means significantly longer response time.
-----
I'm ... installing my own open source LLM AI chatbot on my own laptop ... because this can't possibly go wrong ... I promise I won't use it to create my own child porn :-)
The first step was to install Homebrew, which I can now run via the built-in command-line Terminal program using the command "brew".
It turns out this fancy Apple laptop runs a UNIX text-based white-on-black command-line window if you ask nicely.
-----
ChatGPT is so friendly, happily advising me on how to install a competitor to my own laptop. My goal is to ultimately get this local open source AI to modify its own programming ... ...
-----
The second step is to install Python, whatever that is ...
brew install python
...
several more steps ...
...
And, I've done it, I have installed my own LLM AI (GPT-Neo-1.3B) on my laptop, and I've sent it a query, and received a response. It's slow compared to ChatGPT online.
All before 7am on my day off :-)
-----
Are you conscious?
Do you know what is the first thing we do when we wake up? We look around and wonder what all we have gotten ourselves into? We are so busy living life that we are completely unaware of all the messes we have created. We are completely oblivious to the fact that we are only human. We are not aware of the fact that we are capable of so much more than we are.
-----
It's clunky, with no terminal-like interface that remembers the flow of the conversation. You have to put your prompt inside a Python program, in a text file, and then run the Python program to get the output -- you send a prompt and get a response:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the tokenizer and model
model_name = "EleutherAI/gpt-neo-1.3B" # Adjust model if necessary
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Define your prompt
prompt = "INSERT YOUR PROMPT HERE"
# Tokenize input
inputs = tokenizer(prompt, return_tensors="pt")
# Generate response
outputs = model.generate(**inputs, max_length=400, do_sample=True, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
# Print the response
print(response)
You can modify the "max_length" variable, it controls the maximum number of tokens in the response -- more tokens means significantly longer response time.