TED Vortex (Teodor-Eugen Duțulescu) 0-vortex

Minimal Prompt Chainables

Sequential prompt chaining in one method with context and output back-referencing.

Files

main.py - start here - full example using MinimalChainable from chain.py to build a sequential prompt chain
chain.py - contains zero library minimal prompt chain class
chain_test.py - tests for chain.py, you can ignore this
requirements.py - python requirements

Setup

FAQ on the xz-utils backdoor (CVE-2024-3094)

This is a living document. Everything in this document is made in good faith of being accurate, but like I just said; we don't yet know everything about what's going on.

Update: I've disabled comments as of 2025-01-26 to avoid everyone having notifications for something a year on if someone wants to suggest a correction. Folks are free to email to suggest corrections still, of course.

Background

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

You probably don't know how to do Prompt Engineering

(This post could also be titled "Features missing from most LLM front-ends that should exist")

Apologies for the snarky title, but there has been a huge amount of discussion around so called "Prompt Engineering" these past few months on all kinds of platforms. Much of it is coming from individuals who are peddling around an awful lot of "Prompting" and very little "Engineering".

Most of these discussions are little more than users finding that writing more creative and complicated prompts can help them solve a task that a more simple prompt was unable to help with. I claim this is not Prompt Engineering. This is not to say that crafting good prompts is not a difficult task, but it does not involve doing any kind of sophisticated modifications to general "template" of a prompt.

Others, who I think do deserve to call themselves "Prompt Engineers" (and an awful lot more than that), have been writing about and utilizing the rich new eco-system

Terms

times
- start up time: the time from "the command is executed" to the time "load event is triggered in browser".
- root HMR time: the time from "the root file is changed" to the time "that file is executed in browser".
- leaf HMR time: the time from "the leaf file is changed" to the time "that file is executed in browser".
cold/hot start
- cold start: the dependency optimization cache is deleted before each run
- hot start: the dependency optimization cache exists by each run

Summary

Arch Linux ARM build for M1 (Apple Silicon) VMs

This guide is for building your own Arch Linux ARM VM image and runnig in QEMU, UTM, Parallels...

Preparations in Linux

What you need in Linux phase:

1. qemu-img
2. fdisk
3. kpartx
4. bsdtar

	# Clone llama.cpp
	git clone https://github.com/ggerganov/llama.cpp.git
	cd llama.cpp

	# Build it
	LLAMA_METAL=1 make

	# Download model
	export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin
	wget "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/${MODEL}"

	# Clone llama.cpp
	git clone https://github.com/ggerganov/llama.cpp.git
	cd llama.cpp

	# Build it
	make clean
	LLAMA_METAL=1 make

	# Download model
	export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin

	['absl-py==1.4.0',
	'affine==2.4.0',
	'aiohttp==3.8.1',
	'aiosignal==1.3.1',
	'analytics-python==1.4.post1',
	'anyio==3.7.1',
	'anytree==2.8.0',
	'argcomplete==1.10.3',
	'argon2-cffi-bindings==21.2.0',
	'argon2-cffi==21.3.0',

	# Source: https://gist.github.com/vfarcic/78c1d2a87baf31512b87a2254194b11c

	###############################################################
	# How To Create A Complete Internal Developer Platform (IDP)? #
	# https://youtu.be/Rg98GoEHBd4 #
	###############################################################

	# Additional Info:
	# - DevOps MUST Build Internal Developer Platform (IDP): https://youtu.be/j5i00z3QXyU
	# - How To Create A "Proper" CLI With Shell And Charm Gum: https://youtu.be/U8zCHA-9VLA