S3

The s3 package is an AWS S3 implementation of the golly VFS (Virtual File System) interface. Blank-import it to auto-register the s3:// scheme with the VFS manager.

Installation

go get oss.nandlabs.io/golly-aws/s3

Features

Read / Write — Stream objects from S3; buffered writes flushed on Close()
Create / Open — Create new or open existing objects
Delete / DeleteAll — Delete single objects or recursively under a prefix
Copy / Move — Server-side copy with stream fallback; move = copy + delete
List / Walk / Find — List direct children, recursively walk, or filter with a custom FileFilter
Mkdir / MkdirAll — Create directory markers (zero-byte objects with trailing /)
Metadata — Object info (size, last modified, content type), custom properties
Auto-Registration — Blank import registers the s3:// scheme with vfs.GetManager()
Config Resolution — Per-bucket or global AWS config via awscfg.GetConfig

Quick Start

package main

import (
    "fmt"
    "io"
    "log"

    "oss.nandlabs.io/golly-aws/awscfg"
    _ "oss.nandlabs.io/golly-aws/s3"
    "oss.nandlabs.io/golly/vfs"
)

func main() {
    // Register AWS config
    cfg := awscfg.NewConfig("us-east-1")
    awscfg.Manager.Register("s3", cfg)

    mgr := vfs.GetManager()

    // Write a file
    w, _ := mgr.CreateRaw("s3://my-bucket/hello.txt")
    w.Write([]byte("Hello from Golly!"))
    w.Close()

    // Read it back
    r, _ := mgr.OpenRaw("s3://my-bucket/hello.txt")
    data, _ := io.ReadAll(r)
    r.Close()
    fmt.Println(string(data))

    // List files
    files, _ := mgr.ListRaw("s3://my-bucket/")
    for _, f := range files {
        info, _ := f.Info()
        fmt.Printf("%s (%d bytes)\n", info.Name(), info.Size())
    }
}

Architecture

┌─────────────────────────────────────────────────────────┐
│  import _ "oss.nandlabs.io/golly-aws/s3"                │
│                                                         │
│  vfs.GetManager().OpenRaw("s3://bucket/key")            │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────┐
│  golly/vfs.Manager                                      │
│  Routes to filesystem by URL scheme ("s3")              │
└───────────────────────┬─────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────────────┐
│  s3.S3FS (VFileSystem)                                  │
│  1. parseURL(u) → bucket + key                          │
│  2. getS3Client → awscfg.GetConfig(u, "s3")             │
│  3. S3 API call → PutObject / GetObject / ListV2 etc    │
└─────────────────────────────────────────────────────────┘

URL Format

s3://bucket-name/path/to/object.txt
s3://bucket-name/path/to/folder/

URL	Bucket	Key	Type
`s3://my-bucket/data/report.csv`	`my-bucket`	`data/report.csv`	File
`s3://my-bucket/logs/`	`my-bucket`	`logs/`	Directory

Configuration

Configuration is resolved via awscfg.GetConfig(url, "s3") using a three-step fallback:

url.Host — bucket-specific config (e.g., "my-bucket")
url.Host + "/" + url.Path — path-specific config
Fallback "s3" — default config for all S3 operations

// Per-bucket configuration
awscfg.Manager.Register("s3", defaultCfg)
awscfg.Manager.Register("prod-bucket", prodCfg)

LocalStack / Custom Endpoint

cfg := awscfg.NewConfig("us-east-1")
cfg.SetEndpoint("http://localhost:4566")
cfg.SetStaticCredentials("test", "test", "")
awscfg.Manager.Register("s3", cfg)

Bedrock SNS