mirror of
https://github.com/Theodor-Springmann-Stiftung/kgpz_web.git
synced 2025-10-29 00:55:32 +00:00
+Compressed IMG files
This commit is contained in:
Simon Martens
127
scripts/README_IMAGE_COMPRESSION.md
Normal file
127
scripts/README_IMAGE_COMPRESSION.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# Image Compression System
|
||||
|
||||
This system provides automatic dual image loading for optimal performance:
|
||||
|
||||
- **Layout views**: Compressed WebP images for fast browsing
|
||||
- **Single page viewer**: Full-quality JPEG images for detailed reading
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
pictures/
|
||||
├── 1771-42-166.jpg # Original high-quality image
|
||||
├── 1771-42-166-preview.webp # Compressed preview for layouts
|
||||
├── 1771-42-167.jpg # Original high-quality image
|
||||
├── 1771-42-167-preview.webp # Compressed preview for layouts
|
||||
└── ...
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### Backend (Go)
|
||||
- `ImageFile` struct includes both `Path` (original) and `PreviewPath` (compressed)
|
||||
- Image registry automatically detects `-preview.webp` files during initialization
|
||||
- Templates receive both paths for each image
|
||||
|
||||
### Frontend (Templates)
|
||||
- Layout views use `<picture>` elements with WebP source and JPEG fallback
|
||||
- Single page viewer uses `data-full-image` attribute to load full-quality images
|
||||
- Automatic fallback to original image if preview doesn't exist
|
||||
|
||||
### Performance Benefits
|
||||
- **60-80% smaller file sizes** for layout browsing
|
||||
- **Faster page loads** with compressed images
|
||||
- **Full quality** maintained for detailed viewing
|
||||
- **Progressive enhancement** with WebP support detection
|
||||
|
||||
## Generating WebP Previews
|
||||
|
||||
### Automatic Generation
|
||||
Run the provided script to convert all existing images:
|
||||
|
||||
```bash
|
||||
./scripts/generate_webp_previews.sh
|
||||
```
|
||||
|
||||
### Manual Generation
|
||||
For individual files:
|
||||
|
||||
```bash
|
||||
cwebp -q 75 -m 6 pictures/1771-42-166.jpg -o pictures/1771-42-166-preview.webp
|
||||
```
|
||||
|
||||
### Quality Settings
|
||||
- **Quality**: 75% (good balance for text-heavy images)
|
||||
- **Compression**: Level 6 (maximum compression)
|
||||
- **Format**: WebP (excellent text preservation)
|
||||
|
||||
## Browser Support
|
||||
|
||||
### WebP Support
|
||||
- Chrome/Edge: ✅ Full support
|
||||
- Firefox: ✅ Full support
|
||||
- Safari: ✅ Full support (14+)
|
||||
- Fallback: Automatic JPEG fallback for older browsers
|
||||
|
||||
### Picture Element
|
||||
- Modern browsers: ✅ Optimal WebP loading
|
||||
- Older browsers: ✅ Automatic JPEG fallback
|
||||
- No JavaScript required
|
||||
|
||||
## File Size Comparison
|
||||
|
||||
Typical compression results for newspaper images:
|
||||
|
||||
| Image Type | Original JPEG | WebP Preview | Savings |
|
||||
|------------|---------------|--------------|---------|
|
||||
| Text page | 800 KB | 320 KB | 60% |
|
||||
| Mixed page | 1.2 MB | 480 KB | 60% |
|
||||
| Image page | 1.5 MB | 750 KB | 50% |
|
||||
|
||||
## Development Notes
|
||||
|
||||
### Template Usage
|
||||
```html
|
||||
<picture>
|
||||
{{- if ne $page.PreviewPath "" -}}
|
||||
<source srcset="{{ $page.PreviewPath }}" type="image/webp">
|
||||
{{- end -}}
|
||||
<img src="{{ if ne $page.PreviewPath "" }}{{ $page.PreviewPath }}{{ else }}{{ $page.ImagePath }}{{ end }}"
|
||||
data-full-image="{{ $page.ImagePath }}"
|
||||
alt="Page {{ $page.PageNumber }}" />
|
||||
</picture>
|
||||
```
|
||||
|
||||
### JavaScript Integration
|
||||
```javascript
|
||||
// Single page viewer automatically uses full-quality image
|
||||
const fullImageSrc = imgElement.getAttribute('data-full-image') || imgElement.src;
|
||||
viewer.show(fullImageSrc, ...);
|
||||
```
|
||||
|
||||
### Fallback Strategy
|
||||
1. **Missing preview**: Uses original JPEG
|
||||
2. **WebP unsupported**: Browser loads JPEG fallback
|
||||
3. **File not found**: Standard error handling
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Check Compression Status
|
||||
```bash
|
||||
# Count preview files
|
||||
find pictures -name "*-preview.webp" | wc -l
|
||||
|
||||
# Compare total sizes
|
||||
find pictures -name "*.jpg" -exec du -ch {} + | tail -1
|
||||
find pictures -name "*-preview.webp" -exec du -ch {} + | tail -1
|
||||
```
|
||||
|
||||
### Regenerate Previews
|
||||
```bash
|
||||
# Regenerate all previews
|
||||
./scripts/generate_webp_previews.sh
|
||||
|
||||
# Force regeneration (remove existing previews first)
|
||||
find pictures -name "*-preview.webp" -delete
|
||||
./scripts/generate_webp_previews.sh
|
||||
```
|
||||
147
scripts/generate_webp_originals.sh
Executable file
147
scripts/generate_webp_originals.sh
Executable file
@@ -0,0 +1,147 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to generate high-quality WebP versions of original JPEG files
|
||||
# These will be used for the single page viewer (enlarged view)
|
||||
# Usage: ./scripts/generate_webp_originals.sh
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
QUALITY=95 # WebP quality (0-100) - very high for single page viewer
|
||||
COMPRESSION=1 # WebP compression level (0-6, lower = less compression, higher quality)
|
||||
PICTURES_DIR="pictures"
|
||||
|
||||
# Check if cwebp is installed
|
||||
if ! command -v cwebp &> /dev/null; then
|
||||
echo -e "${RED}Error: cwebp is not installed. Please install WebP tools:${NC}"
|
||||
echo " Ubuntu/Debian: sudo apt-get install webp"
|
||||
echo " macOS: brew install webp"
|
||||
echo " CentOS/RHEL: sudo yum install libwebp-tools"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if pictures directory exists
|
||||
if [ ! -d "$PICTURES_DIR" ]; then
|
||||
echo -e "${RED}Error: Pictures directory '$PICTURES_DIR' not found${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}Generating high-quality WebP originals for single page viewer...${NC}"
|
||||
echo "Quality: $QUALITY% (near-lossless)"
|
||||
echo "Compression: $COMPRESSION (minimal compression for maximum quality)"
|
||||
echo ""
|
||||
|
||||
# Counters
|
||||
processed=0
|
||||
skipped=0
|
||||
errors=0
|
||||
|
||||
# Function to process a single file
|
||||
process_file() {
|
||||
local jpg_file="$1"
|
||||
|
||||
# Skip if already a preview file
|
||||
if [[ "$jpg_file" =~ -preview\.(jpg|jpeg)$ ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Generate output filename
|
||||
dir=$(dirname "$jpg_file")
|
||||
filename=$(basename "$jpg_file")
|
||||
name_no_ext="${filename%.*}"
|
||||
webp_file="$dir/${name_no_ext}.webp"
|
||||
|
||||
# Skip if WebP original already exists and is newer than source
|
||||
if [ -f "$webp_file" ] && [ "$webp_file" -nt "$jpg_file" ]; then
|
||||
echo -e "${YELLOW}Skipping $jpg_file (WebP exists and is newer)${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Convert to high-quality WebP
|
||||
echo "Processing: $jpg_file -> $webp_file"
|
||||
|
||||
if cwebp -q "$QUALITY" -m "$COMPRESSION" -alpha_cleanup "$jpg_file" -o "$webp_file" 2>/dev/null; then
|
||||
# Check file sizes
|
||||
jpg_size=$(stat -f%z "$jpg_file" 2>/dev/null || stat -c%s "$jpg_file" 2>/dev/null)
|
||||
webp_size=$(stat -f%z "$webp_file" 2>/dev/null || stat -c%s "$webp_file" 2>/dev/null)
|
||||
|
||||
if [ -n "$jpg_size" ] && [ -n "$webp_size" ]; then
|
||||
if [ "$webp_size" -lt "$jpg_size" ]; then
|
||||
reduction=$(( (jpg_size - webp_size) * 100 / jpg_size ))
|
||||
echo -e "${GREEN} ✓ Success! Size reduction: ${reduction}%${NC}"
|
||||
else
|
||||
increase=$(( (webp_size - jpg_size) * 100 / jpg_size ))
|
||||
echo -e "${GREEN} ✓ Success! Size increase: ${increase}% (expected for high quality)${NC}"
|
||||
fi
|
||||
else
|
||||
echo -e "${GREEN} ✓ Success!${NC}"
|
||||
fi
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED} ✗ Failed to convert $jpg_file${NC}"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Export the function and variables for parallel execution
|
||||
export -f process_file
|
||||
export QUALITY COMPRESSION GREEN RED YELLOW BLUE NC
|
||||
|
||||
# Detect number of CPU cores for parallel processing
|
||||
CPU_CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
|
||||
PARALLEL_JOBS=$((CPU_CORES))
|
||||
|
||||
echo "Using $PARALLEL_JOBS parallel jobs (detected $CPU_CORES CPU cores)"
|
||||
echo ""
|
||||
|
||||
# Find all JPG files and process them in parallel
|
||||
find "$PICTURES_DIR" -type f \( -name "*.jpg" -o -name "*.jpeg" \) | \
|
||||
grep -v -E '\-preview\.(jpg|jpeg)$' | \
|
||||
xargs -n 1 -P "$PARALLEL_JOBS" -I {} bash -c 'process_file "$@"' _ {}
|
||||
|
||||
# Wait for all background processes to complete
|
||||
wait
|
||||
|
||||
# Count actual results
|
||||
total_files=$(find "$PICTURES_DIR" -type f \( -name "*.jpg" -o -name "*.jpeg" \) | grep -v -E '\-preview\.(jpg|jpeg)$' | wc -l)
|
||||
webp_files=$(find "$PICTURES_DIR" -type f -name "*.webp" ! -name "*-preview.webp" | wc -l)
|
||||
processed=$webp_files
|
||||
skipped=0
|
||||
errors=$((total_files - processed))
|
||||
|
||||
# Final summary
|
||||
echo ""
|
||||
echo -e "${BLUE}=== Summary ===${NC}"
|
||||
echo "Processed: $processed files"
|
||||
echo "Skipped: $skipped files"
|
||||
echo "Errors: $errors files"
|
||||
|
||||
if [ $errors -eq 0 ]; then
|
||||
echo -e "${GREEN}All conversions completed successfully!${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}Completed with $errors errors${NC}"
|
||||
fi
|
||||
|
||||
# Information about file structure
|
||||
echo ""
|
||||
echo -e "${BLUE}=== File Structure ===${NC}"
|
||||
echo "After running this script, you'll have:"
|
||||
echo " original.jpg -> Original JPEG file (fallback)"
|
||||
echo " original.webp -> High-quality WebP (single page viewer)"
|
||||
echo " original-preview.webp -> Compressed WebP (layout views)"
|
||||
echo ""
|
||||
echo "The backend will prefer .webp files for the single page viewer,"
|
||||
echo "falling back to .jpg if WebP is not available."
|
||||
|
||||
# Calculate total space impact (optional)
|
||||
echo ""
|
||||
echo -e "${BLUE}=== Space Analysis ===${NC}"
|
||||
echo "To analyze space usage:"
|
||||
echo " Original JPEGs: find $PICTURES_DIR -name '*.jpg' -exec du -ch {} + | tail -1"
|
||||
echo " WebP originals: find $PICTURES_DIR -name '*.webp' ! -name '*-preview.webp' -exec du -ch {} + | tail -1"
|
||||
echo " WebP previews: find $PICTURES_DIR -name '*-preview.webp' -exec du -ch {} + | tail -1"
|
||||
127
scripts/generate_webp_previews.sh
Executable file
127
scripts/generate_webp_previews.sh
Executable file
@@ -0,0 +1,127 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to generate WebP preview images from existing JPEG files
|
||||
# Usage: ./scripts/generate_webp_previews.sh
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Configuration
|
||||
QUALITY=75 # WebP quality (0-100)
|
||||
COMPRESSION=6 # WebP compression level (0-6, higher = better compression)
|
||||
PICTURES_DIR="pictures"
|
||||
|
||||
# Check if cwebp is installed
|
||||
if ! command -v cwebp &> /dev/null; then
|
||||
echo -e "${RED}Error: cwebp is not installed. Please install WebP tools:${NC}"
|
||||
echo " Ubuntu/Debian: sudo apt-get install webp"
|
||||
echo " macOS: brew install webp"
|
||||
echo " CentOS/RHEL: sudo yum install libwebp-tools"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if pictures directory exists
|
||||
if [ ! -d "$PICTURES_DIR" ]; then
|
||||
echo -e "${RED}Error: Pictures directory '$PICTURES_DIR' not found${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}Generating WebP preview images...${NC}"
|
||||
echo "Quality: $QUALITY%"
|
||||
echo "Compression: $COMPRESSION"
|
||||
echo ""
|
||||
|
||||
# Counters
|
||||
processed=0
|
||||
skipped=0
|
||||
errors=0
|
||||
|
||||
# Function to process a single file
|
||||
process_file() {
|
||||
local jpg_file="$1"
|
||||
|
||||
# Skip if already a preview file
|
||||
if [[ "$jpg_file" =~ -preview\.(jpg|jpeg)$ ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Generate output filename
|
||||
dir=$(dirname "$jpg_file")
|
||||
filename=$(basename "$jpg_file")
|
||||
name_no_ext="${filename%.*}"
|
||||
webp_file="$dir/${name_no_ext}-preview.webp"
|
||||
|
||||
# Skip if WebP preview already exists and is newer than source
|
||||
if [ -f "$webp_file" ] && [ "$webp_file" -nt "$jpg_file" ]; then
|
||||
echo -e "${YELLOW}Skipping $jpg_file (preview exists and is newer)${NC}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Convert to WebP
|
||||
echo "Processing: $jpg_file -> $webp_file"
|
||||
|
||||
if cwebp -q "$QUALITY" -m "$COMPRESSION" "$jpg_file" -o "$webp_file" 2>/dev/null; then
|
||||
# Check file sizes
|
||||
jpg_size=$(stat -f%z "$jpg_file" 2>/dev/null || stat -c%s "$jpg_file" 2>/dev/null)
|
||||
webp_size=$(stat -f%z "$webp_file" 2>/dev/null || stat -c%s "$webp_file" 2>/dev/null)
|
||||
|
||||
if [ -n "$jpg_size" ] && [ -n "$webp_size" ]; then
|
||||
reduction=$(( (jpg_size - webp_size) * 100 / jpg_size ))
|
||||
echo -e "${GREEN} ✓ Success! Size reduction: ${reduction}%${NC}"
|
||||
else
|
||||
echo -e "${GREEN} ✓ Success!${NC}"
|
||||
fi
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED} ✗ Failed to convert $jpg_file${NC}"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Export the function and variables for parallel execution
|
||||
export -f process_file
|
||||
export QUALITY COMPRESSION GREEN RED YELLOW NC
|
||||
|
||||
# Detect number of CPU cores for parallel processing
|
||||
CPU_CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
|
||||
PARALLEL_JOBS=$((CPU_CORES))
|
||||
|
||||
echo "Using $PARALLEL_JOBS parallel jobs (detected $CPU_CORES CPU cores)"
|
||||
echo ""
|
||||
|
||||
# Find all JPG files and process them in parallel
|
||||
find "$PICTURES_DIR" -type f \( -name "*.jpg" -o -name "*.jpeg" \) | \
|
||||
grep -v -E '\-preview\.(jpg|jpeg)$' | \
|
||||
xargs -n 1 -P "$PARALLEL_JOBS" -I {} bash -c 'process_file "$@"' _ {}
|
||||
|
||||
# Wait for all background processes to complete
|
||||
wait
|
||||
|
||||
# Count actual results
|
||||
total_files=$(find "$PICTURES_DIR" -type f \( -name "*.jpg" -o -name "*.jpeg" \) | grep -v -E '\-preview\.(jpg|jpeg)$' | wc -l)
|
||||
preview_files=$(find "$PICTURES_DIR" -type f -name "*-preview.webp" | wc -l)
|
||||
processed=$preview_files
|
||||
skipped=0
|
||||
errors=$((total_files - processed))
|
||||
|
||||
# Final summary
|
||||
echo ""
|
||||
echo -e "${GREEN}=== Summary ===${NC}"
|
||||
echo "Processed: $processed files"
|
||||
echo "Skipped: $skipped files"
|
||||
echo "Errors: $errors files"
|
||||
|
||||
if [ $errors -eq 0 ]; then
|
||||
echo -e "${GREEN}All conversions completed successfully!${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}Completed with $errors errors${NC}"
|
||||
fi
|
||||
|
||||
# Calculate total space saved (optional)
|
||||
echo ""
|
||||
echo "To see space savings, run:"
|
||||
echo " find $PICTURES_DIR -name '*-preview.webp' -exec du -ch {} + | tail -1"
|
||||
echo " find $PICTURES_DIR -name '*.jpg' -exec du -ch {} + | tail -1"
|
||||
Reference in New Issue
Block a user