jashton/one-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated documentation and scripts.

Browse Source
This commit is contained in:
Violet Ash
2024-11-01 14:50:14 -06:00
parent cdac0e0ec8
commit 0a40a0f472
6 changed files with 136 additions and 116 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+68 -46
View File
@@ -1,64 +1,64 @@
One AI - Name TBD, this is just the shortest name I could think of and IDRC
Usage:
As of right now, running is not streamlined. Of particular note is using
cloudflared tunnels. I've done my own Cloudflare Tunnel configuration,
but in the future that will be automated as part of the installation process,
along with much more. These instructions are (currently) only for Linux.
In the meantime:
INSTALLATION:
1. Install dependencies listed below:
- ollama,
- go,
- cloudflared,
- flutter,
1. Install and configure dependencies. On Arch Linux, run the following:
```
# Install dependencies
sudo pacman -S ollama go cloudflared docker
yay -S flutter-bin
- docker, (optional)
# Configure ollama
ollama pull llama3
#ollama pull qwen:0.5b # Use this if hosting on older hardware.
eg. `pacman -S ollama go docker cloudflared; yay -s flutter;`
# Configure cloudflared
cloudflared tunnel login
2. Setup Ollama. If you're on a laptop (non-gaming variety), run `ollama pull
qwen:0.5b`. If you're running this on something with a beefy GPU, run
`ollama pull llama3`. Word of warning, qwen is really dumb, but it's light.
it tried to convince me there were 12 letters in the alphabet.
3. Start the web server via Docker or plain Golang:
a) Go to ./backend/src/security-layer/ and run `./init`. The exact script is
within the repository, but the commands can be ran as follows:
# Configure Docker
systemctl start docker
#systemctl enable docker
```
#!/bin/zsh
# Force delete the old container and re-initialize the container.
sudo docker rm -f security-layer
# Build the Docker image.
sudo docker build --tag security-layer:latest .
2. Run the installation script. This will create, configure, and start the
Cloudflare Tunnel via cloudflared and create the initial configuration
file for One AI;
# Create a Docker container from the image and connect host port 8000 to container port 8000.
sudo docker run --name security-layer -d -p 8000:8000 security-layer:latest
```
3. Start the server via Docker or plain Golang:
a) Go to ./backend/src/security-layer/ and run `./init`.
b) Go to ./backend/src/security-layer/container and run `go run router.go`.
4. Edit ~/.cloudflared/config.yml to to indicate something along these lines:
```
tunnel: another-long-weird-hash
credentials-file: /home/username/.cloudflared/another-long-weird-hash.json
ingress:
- hostname: somedomain.cloudflare.com
service: http://localhost:8080
```
By default, this will expose 4 ports: the router at port 1111, the API on
1112, Auth/Session Management on 1113, and a static webpage on 1114. The
ports and webpage directory are configurable through
`~/.config/one-ai/test.json`.
5. Start the Cloudflare Tunnel on the host/server machine. For Linux, it should
be a command like `sudo cloudflared service install areallylonghash@sha-256?`
CONFIGURATION:
The primary configuration file is located at `~/.config/one-ai/`. It is
recommended to keep two files, `test.json` and `live.json`. This is purely
for security purposes.
The following are configurable parameters:
- `text_model`: Specify the LLM used through Ollama, eg. "llama3".
- `response_stream`: `true` or `false`, to receive communication as it is
generated or once it is completed generating.
- `domain`: The URL to be used to access the router.
- `router_port`: The port to be exposed to the internet.
- `api`: The URL to be used to access the One AI API.
- `api_port`: A port to be only accessible to the localhost.
- `auth_port`: A port to be only accessible to the localhost.
- `webpage_dir`: The directory containing a website.
- `webpage_port`: The port to expose the webpage directory to the internet.
TODO:
Urgent:
- [x] Basic Documentation
- [ ] Installation/setup script
- [x] Installation/setup script
- [ ] AES-256 Private and Public key handshake via USB.
- [ ] Cloudflare Tunnel setup via cloudflared
- [x] Cloudflare Tunnel setup via cloudflared
- [ ] Daemonize service
- [x] Encryption
@@ -66,8 +66,9 @@ TODO:
- [x] Encrypt API request
- [x] Encrypt API response
- [ ] Dockerize backend
- [ ] Broke Dockerfile, need to expose additional ports.
- [x] Dockerize backend
- [x] Broke Dockerfile, need to expose additional ports.
- [ ] Dynamically expose ports based on primary configuration file.
- [ ] Golang backend, connect to other services/projects
- [x] Port 8080 -> self-hosted portfolio
@@ -81,7 +82,7 @@ TODO:
- [ ] Connect Flutter Web App to server
Soon:
Later:
- [ ] Raspberry Pi nightly package
- [ ] Image support via llava
@@ -99,7 +100,28 @@ TODO:
- [x] On-device Speech to Text
- [x] On-device Text to Speech
Later:
router.go:
- [ ] Allow for additional fields for the client.
- [ ] Add preprocessing based upon filetypes.
- [ ] Add support for dynamically changing models.
- [ ] Actually make text streaming a toggleable setting.
- [ ] Authentication via JWT.
- [ ] Get config via CLI argument or environment variable.
- [ ] Add --test CLI argument when testing solely on localhost.
- [ ] Error handling, testing, & documentation.
authentication.go:
- [ ] 32 bit key generation & storage between server and client
- [ ] Add testing flag
- [ ] Error handling, testing, & documentation.
serveAPI.go:
- [ ] Error handling, testing, & documentation.
servePage.go:
- [ ] Error handling, testing, & documentation.
Much Later:
- [ ] Document support via OmniParser
- [ ] Encode files to Base64 to/from backend.
- [ ] Connect to external services like Google Drive
backend/install.sh
+8 -5
View File
@@ -11,7 +11,7 @@ VERBOSE=0
while getopts "d:p:n?s?c?v?h?" opt; do
case "$opt" in
h|\?)
echo "A utility script for quickly getting up and started with self-hosted applicaitons."
echo "A utility script for quickly getting up and started with self-hosted applications."
echo "This script assumes the following:"
echo "- You have already signed in to the cloudflared CLI application;"
echo "- You own a domain through Cloudflare;"
@@ -40,11 +40,13 @@ done
shift $((OPTIND-1))
[ "${1:-}" = "--" ] && shift
# TODO: Install dependencies.
# TODO: Add CLI option to enable cloudflared
# TODO: Connect with cross-platform package manager from quaxlyqueen/scripts
# TODO: Setup ollama (pull LLM best fitting hardware).
createTunnel
# Initialize cloudflared
#cloudflared tunnel login
createTunnel() {
if [[ "$VERBOSE" -eq 1 ]]; then
echo "Creating cloudflared tunnel..."
fi
@@ -95,8 +97,9 @@ do
cloudflared tunnel route dns $TUNNEL_ID ${SUBDOMAINS[$i]}.$DOMAIN
done
cloudflared tunnel route dns $TUNNEL_ID $DOMAIN
cloudflared tunnel run $TUNNEL_NAME
bg cloudflared tunnel run $TUNNEL_NAME
if [[ "$VERBOSE" -eq 1 ]]; then
echo "Complete!"
fi
}
backend/src/security-layer/Dockerfile
+5 -1
View File
@@ -21,7 +21,11 @@ RUN CGO_ENABLED=0 GOOS=linux go build -o /security-layer
# But we can document in the Dockerfile what ports
# the application is going to listen on by default.
# https://docs.docker.com/reference/dockerfile/#expose
EXPOSE 8000
# TODO: Need to dynamically expose ports based upon ~/.config/one-ai/test.json
EXPOSE 1111
EXPOSE 1112
EXPOSE 1113
EXPOSE 1114
# Run
CMD ["/security-layer"]
backend/src/security-layer/container/backend
Executable
BIN
View File
Binary file not shown.
backend/src/security-layer/init
+2 -2
View File
@@ -5,5 +5,5 @@ sudo docker rm -f security-layer
# Build the Docker image.
sudo docker build --tag security-layer:latest .
# Create a Docker container from the image and connect host port 8000 to container port 8000.
sudo docker run --name security-layer -d -p 8000:8000 security-layer:latest
# Create a Docker container from the image and connect host port 1111 to container port 1111, and 1114 to 1114.
sudo docker run --name security-layer -d -p 1111:1111 -p 1114:1114 security-layer:latest
backend/test.sh
-9
View File
@@ -1,9 +0,0 @@
SUB=("api=1" "test=2" "portfolio=3")
arraylength=${#SUB[@]}
for (( i=0; i<${arraylength}; i++ ));
do
test="$(echo ${SUB[$i]} | cut -f1 -d=)"
test2="$(echo ${SUB[$i]} | cut -f2 -d=)"
echo $test $test2
done