😇

gapsong

😇

Exploring LLM Quantization with XAI. Feel free to check out my repositories and contact me for more information.

wassname / hf_perplexity.py

Last active March 31, 2025 23:08

simple perplexity for huggingface models similar to llam..cpp

	# Directly taken from https://huggingface.co/spaces/evaluate-measurement/perplexity/blob/main/perplexity.py
	# TODO replace with a strided version https://github.com/huggingface/transformers/issues/9648#issuecomment-812981524
	import numpy as np
	import torch
	import itertools
	from torch.nn import CrossEntropyLoss
	from tqdm.auto import tqdm
	import torch.nn.functional as F
	from datasets import load_dataset, Dataset
	from transformers import AutoTokenizer, AutoModelForCausalLM

ChrisHayduk / merge_qlora_with_quantized_model.py

Last active September 27, 2025 08:22

Merging QLoRA weights with quantized model

	"""

	The code below combines approaches published by both @eugene-yh and @jinyongyoo on Github.

	Thanks for the contributions guys!

	"""

	import torch
	import peft

SunMarc / finetune_llama_gptq.py

Last active October 12, 2025 03:18

Finetune GPTQ model with peft and tlr

	# coding=utf-8
	# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
	#
	# Licensed under the Apache License, Version 2.0 (the "License");
	# you may not use this file except in compliance with the License.
	# You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0
	#
	# Unless required by applicable law or agreed to in writing, software