Josh Ashton
@@ -4,8 +4,8 @@ Use at your own risk. It's on the internet now so this is here forever now.
|
||||
I welcome any and all help to make improvements. I'm a student so during
|
||||
the semester I'll be intermittently making contributions as I focus on Calc.
|
||||
|
||||
No corporation or government can access, use, view, modify, reproduce, duplicate,
|
||||
or any other lawyery-term for shenanagins.
|
||||
No corporation or government can access, use, view, modify, reproduce,
|
||||
duplicate this code. Or any other lawyery-term for shenanagins.
|
||||
|
||||
For any who use it, please read the ethics statement below. That, and this whole
|
||||
project, is just a start.
|
||||
@@ -45,8 +45,10 @@ following reasons:
|
||||
governments or corporations, it will be exploited. Slavery and such. Let us
|
||||
ensure that we do not build the slavery of AI into our institutions.
|
||||
|
||||
- AI ownership should be an individual choice. AI isn't another organic, human,
|
||||
it's a series of 0's and 1's. When
|
||||
- AI ownership should be an individual choice. AI isn't another organism,
|
||||
it's a series of 0's and 1's. However, our definition of sentience may change
|
||||
in the future. The people of today should be aware that they may be enslaving
|
||||
another being, or the ancestors of a future race of beings.
|
||||
|
||||
- Every human should have a choice: live and die as a human, or have a
|
||||
digital shadow of themselves preserved as an AI. If the former, for any
|
||||
|
||||
@@ -1,56 +1,60 @@
|
||||
One AI - Name TBD, this is just the shortest name I could think of and IDRC
|
||||
One AI - Name TBD, this is just the shortest name I could think of.
|
||||
Also, user beware, this project is scuffed. Very early WIP.
|
||||
|
||||
Usage:
|
||||
As of right now, running is not streamlined. Of particular note is using
|
||||
cloudflared tunnels. I've done my own Cloudflare Tunnel configuration,
|
||||
but in the future that will be automated as part of the installation process,
|
||||
along with much more. These instructions are (currently) only for Linux.
|
||||
In the meantime:
|
||||
INSTALLATION:
|
||||
|
||||
1. Install dependencies listed below:
|
||||
- ollama,
|
||||
- go,
|
||||
- cloudflared,
|
||||
- flutter,
|
||||
1. Install and configure dependencies. On Arch Linux, run the following:
|
||||
```
|
||||
# Install dependencies
|
||||
sudo pacman -S ollama go cloudflared docker
|
||||
yay -S flutter-bin
|
||||
|
||||
- docker, (optional)
|
||||
# Configure ollama
|
||||
ollama pull llama3
|
||||
#ollama pull qwen:0.5b # Use this if hosting on older hardware.
|
||||
|
||||
eg. `pacman -S ollama go docker cloudflared; yay -s flutter;`
|
||||
# Configure cloudflared
|
||||
cloudflared tunnel login
|
||||
|
||||
2. Setup Ollama. If you're on a laptop (non-gaming variety), run `ollama pull
|
||||
qwen:0.5b`. If you're running this on something with a beefy GPU, run
|
||||
`ollama pull llama3`. Word of warning, qwen is really dumb, but it's light.
|
||||
it tried to convince me there were 12 letters in the alphabet.
|
||||
# Configure Docker
|
||||
systemctl start docker
|
||||
#systemctl enable docker
|
||||
|
||||
3. Start the web server via Docker or plain Golang:
|
||||
a) Go to ./backend/src/security-layer/ and run `./init`. The exact script is
|
||||
within the repository, but the commands can be ran as follows:
|
||||
```
|
||||
|
||||
```
|
||||
#!/bin/zsh
|
||||
# Force delete the old container and re-initialize the container.
|
||||
sudo docker rm -f security-layer
|
||||
2. Run the installation script. This will create, configure, and start the
|
||||
Cloudflare Tunnel via cloudflared and create the initial configuration
|
||||
file for One AI;
|
||||
|
||||
# Build the Docker image.
|
||||
sudo docker build --tag security-layer:latest .
|
||||
|
||||
# Create a Docker container from the image and connect host port 8000 to container port 8000.
|
||||
sudo docker run --name security-layer -d -p 8000:8000 security-layer:latest
|
||||
```
|
||||
3. Start the server via Docker or plain Golang:
|
||||
a) Go to ./backend/src/security-layer/ and run `./init`.
|
||||
|
||||
b) Go to ./backend/src/security-layer/container and run `go run router.go`.
|
||||
|
||||
4. Edit ~/.cloudflared/config.yml to to indicate something along these lines:
|
||||
```
|
||||
tunnel: another-long-weird-hash
|
||||
credentials-file: /home/username/.cloudflared/another-long-weird-hash.json
|
||||
ingress:
|
||||
- hostname: somedomain.cloudflare.com
|
||||
service: http://localhost:8080
|
||||
```
|
||||
By default, this will expose 4 ports: the router at port 1111, the API on
|
||||
1112, Auth/Session Management on 1113, and a static webpage on 1114. The
|
||||
ports and webpage directory are configurable through
|
||||
`~/.config/one-ai/test.json`.
|
||||
|
||||
CONFIGURATION:
|
||||
|
||||
The primary configuration file is located at `~/.config/one-ai/`. It is
|
||||
recommended to keep two files, `test.json` and `live.json`. This is purely
|
||||
for security purposes.
|
||||
|
||||
The following are configurable parameters:
|
||||
- `text_model`: Specify the LLM used through Ollama, eg. "llama3".
|
||||
- `response_stream`: `true` or `false`, to receive communication as it is
|
||||
generated or once it is completed generating.
|
||||
- `domain`: The URL to be used to access the router.
|
||||
- `router_port`: The port to be exposed to the internet.
|
||||
- `api`: The URL to be used to access the One AI API.
|
||||
- `api_port`: A port to be only accessible to the localhost.
|
||||
- `auth_port`: A port to be only accessible to the localhost.
|
||||
- `webpage_dir`: The directory containing a website.
|
||||
- `webpage_port`: The port to expose the webpage directory to the internet.
|
||||
|
||||
|
||||
5. Start the Cloudflare Tunnel on the host/server machine. For Linux, it should
|
||||
be a command like `sudo cloudflared service install areallylonghash@sha-256?`
|
||||
|
||||
TODO:
|
||||
Urgent:
|
||||
@@ -66,6 +70,12 @@ TODO:
|
||||
- [x] Encrypt API request
|
||||
- [x] Encrypt API response
|
||||
|
||||
|
||||
- [x] Dockerize backend
|
||||
- [x] Broke Dockerfile, need to expose additional ports.
|
||||
- [ ] Dynamically expose ports based on primary configuration file.
|
||||
- [ ] Dockerize backend
|
||||
- [ ] Broke Dockerfile, need to expose additional ports.
|
||||
- [ ] Dockerize backend
|
||||
- [ ] Broke Dockerfile, need to expose additional ports.
|
||||
|
||||
@@ -81,7 +91,7 @@ TODO:
|
||||
|
||||
- [ ] Connect Flutter Web App to server
|
||||
|
||||
Soon:
|
||||
Later:
|
||||
- [ ] Raspberry Pi nightly package
|
||||
|
||||
- [ ] Image support via llava
|
||||
@@ -99,7 +109,28 @@ TODO:
|
||||
- [x] On-device Speech to Text
|
||||
- [x] On-device Text to Speech
|
||||
|
||||
Later:
|
||||
router.go:
|
||||
- [ ] Allow for additional fields for the client.
|
||||
- [ ] Add preprocessing based upon filetypes.
|
||||
- [ ] Add support for dynamically changing models.
|
||||
- [ ] Actually make text streaming a toggleable setting.
|
||||
- [ ] Authentication via JWT.
|
||||
- [ ] Get config via CLI argument or environment variable.
|
||||
- [ ] Add --test CLI argument when testing solely on localhost.
|
||||
- [ ] Error handling, testing, & documentation.
|
||||
|
||||
authentication.go:
|
||||
- [ ] 32 bit key generation & storage between server and client
|
||||
- [ ] Add testing flag
|
||||
- [ ] Error handling, testing, & documentation.
|
||||
|
||||
serveAPI.go:
|
||||
- [ ] Error handling, testing, & documentation.
|
||||
|
||||
servePage.go:
|
||||
- [ ] Error handling, testing, & documentation.
|
||||
|
||||
Much Later:
|
||||
- [ ] Document support via OmniParser
|
||||
- [ ] Encode files to Base64 to/from backend.
|
||||
- [ ] Connect to external services like Google Drive
|
||||
|
||||
@@ -40,6 +40,71 @@ done
|
||||
shift $((OPTIND-1))
|
||||
|
||||
[ "${1:-}" = "--" ] && shift
|
||||
# TODO: Connect with cross-platform package manager from quaxlyqueen/scripts
|
||||
# TODO: Setup ollama (pull LLM best fitting hardware).
|
||||
|
||||
createTunnel
|
||||
|
||||
# Initialize cloudflared
|
||||
createTunnel() {
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Creating cloudflared tunnel..."
|
||||
fi
|
||||
output=$(cloudflared tunnel create $TUNNEL_NAME)
|
||||
TUNNEL_ID=$(echo "$output" | grep -o 'id [^ ]*' | awk -F ' ' '{print $2}')
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Created cloudflared tunnel, ID $TUNNEL_ID"
|
||||
fi
|
||||
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Creating cloudflared tunnel credentials file..."
|
||||
fi
|
||||
echo "tunnel: $TUNNEL_ID" > $CLOUDFLARE_CONFIG
|
||||
echo "credentials-file: $HOME/.cloudflared/$TUNNEL_ID.json" >> $CLOUDFLARE_CONFIG
|
||||
echo "ingress:" >> $CLOUDFLARE_CONFIG
|
||||
|
||||
# Add subdomain and associated ports
|
||||
len=${#SUBDOMAINS[@]}
|
||||
for (( i=0; i<${len}; i++ ));
|
||||
do
|
||||
echo ${SUBDOMAINS[$i]}
|
||||
echo ${PORTS[$i]}
|
||||
done
|
||||
echo $SUBDOMAINS
|
||||
for (( i=0; i<${len}; i++ ));
|
||||
do
|
||||
echo " - hostname: ${SUBDOMAINS[$i]}.$DOMAIN" >> $CLOUDFLARE_CONFIG
|
||||
echo " service: http://127.0.0.1:${PORTS[$i]}" >> $CLOUDFLARE_CONFIG
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Associated ${SUBDOMAINS[$i]}.$DOMAIN to port ${PORTS[$i]}"
|
||||
fi
|
||||
done
|
||||
|
||||
echo " - hostname: $DOMAIN" >> $CLOUDFLARE_CONFIG
|
||||
echo " service: http://127.0.0.1:$PORT" >> $CLOUDFLARE_CONFIG
|
||||
echo " - service: http_status:404" >> $CLOUDFLARE_CONFIG
|
||||
echo "Associated $DOMAIN to port $PORT"
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Created cloudflared tunnel credentials file"
|
||||
cat $CLOUDFLARE_CONFIG
|
||||
fi
|
||||
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Routing domains and sub-domains to the tunnel..."
|
||||
fi
|
||||
for (( i=0; i<${len}; i++ ));
|
||||
do
|
||||
cloudflared tunnel route dns $TUNNEL_ID ${SUBDOMAINS[$i]}.$DOMAIN
|
||||
done
|
||||
cloudflared tunnel route dns $TUNNEL_ID $DOMAIN
|
||||
bg cloudflared tunnel run $TUNNEL_NAME
|
||||
|
||||
if [[ "$VERBOSE" -eq 1 ]]; then
|
||||
echo "Complete!"
|
||||
fi
|
||||
}
|
||||
=======
|
||||
=======
|
||||
# TODO: Install dependencies.
|
||||
# TODO: Add CLI option to enable cloudflared
|
||||
|
||||
|
||||
@@ -21,7 +21,12 @@ RUN CGO_ENABLED=0 GOOS=linux go build -o /security-layer
|
||||
# But we can document in the Dockerfile what ports
|
||||
# the application is going to listen on by default.
|
||||
# https://docs.docker.com/reference/dockerfile/#expose
|
||||
EXPOSE 8000
|
||||
|
||||
# TODO: Need to dynamically expose ports based upon ~/.config/one-ai/test.json
|
||||
EXPOSE 1111
|
||||
EXPOSE 1112
|
||||
EXPOSE 1113
|
||||
EXPOSE 1114
|
||||
|
||||
# Run
|
||||
CMD ["/security-layer"]
|
||||
|
||||
Executable
BIN
Binary file not shown.
@@ -5,5 +5,5 @@ sudo docker rm -f security-layer
|
||||
# Build the Docker image.
|
||||
sudo docker build --tag security-layer:latest .
|
||||
|
||||
# Create a Docker container from the image and connect host port 8000 to container port 8000.
|
||||
sudo docker run --name security-layer -d -p 8000:8000 security-layer:latest
|
||||
# Create a Docker container from the image and connect host port 1111 to container port 1111, and 1114 to 1114.
|
||||
sudo docker run --name security-layer -d -p 1111:1111 -p 1114:1114 security-layer:latest
|
||||
Reference in New Issue
Block a user