Skip to content
JulioVianaDevPublic

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

go-cloud — Google Drive Streaming Proxy

A Go backend that acts as a streaming proxy between your frontend apps (React web + React Native mobile) and Google Drive. Files are never stored on your server — bytes flow directly from Google to the client (and vice versa), using only ~32KB of server memory regardless of file size.

Why does this exist?

The problem: your app needs to serve files stored in Google Drive, but you don't want to:

  • Store files on your own server (storage costs, disk management, sync headaches)
  • Expose Google API credentials to the frontend
  • Pay for egress bandwidth from a storage bucket you manage
  • Manage file replication, backups, or CDN for your own storage

The solution: use Google Drive as your storage and your Go backend as a zero-storage pipe.

┌──────────┐         ┌──────────┐         ┌──────────────┐
│ React /  │  HTTP   │    Go    │  API    │ Google Drive  │
│ React    │ ──────► │  Proxy   │ ──────► │ (your files)  │
│ Native   │ ◄────── │ (~32KB)  │ ◄────── │              │
└──────────┘ stream  └──────────┘ stream  └──────────────┘

The Go server never writes files to disk. It uses io.Copy which reads chunks from one stream and writes them to another — the default buffer is 32KB. A user downloading a 2GB file through this proxy costs your server exactly 32KB of RAM.


Table of Contents


How It Works

Download (GET /files/download)

1. Frontend sends:       GET /files/download?session=abc&fileId=xyz
2. Go calls:             Google Drive API → Files.Get(fileId).Download()
3. Google returns:       io.ReadCloser (a stream, not the whole file)
4. Go pipes:             io.Copy(httpResponseWriter, googleStream)
5. Frontend receives:    Chunked bytes, with Content-Length header for progress

io.Copy uses a 32KB buffer internally. It reads 32KB from Google, writes 32KB to the client, reads the next 32KB, and so on. The file never exists in Go's memory as a whole.

Upload (POST /files/upload)

1. Frontend sends:       POST /files/upload?session=abc&name=report.pdf
                         Body: raw file bytes
2. Go receives:          r.Body (an io.ReadCloser — also a stream)
3. Go calls:             Files.Create(metadata).Media(r.Body)
4. Google reads:         Chunks directly from the HTTP request body

Same pattern in reverse. The request body (r.Body) is a stream. Go passes it directly to the Google Drive upload API. The file flows from the client through Go into Google without Go ever buffering it.

Delete (DELETE /files/delete)

1. Frontend sends:       DELETE /files/delete?session=abc&fileId=xyz
2. Go calls:             Files.Delete(fileId).Do()
3. Done.                 No body, no stream, no memory.

List (GET /files)

1. Frontend sends:       GET /files?session=abc&pageSize=20
2. Go calls:             Files.List().Q("trashed=false").Do()
3. Go returns:           JSON with file metadata (id, name, size, mimeType, dates)

Paginated via pageToken / nextPageToken. Only metadata is transferred — no file content.


Project Structure

go-cloud/
├── backend/                  # Go streaming proxy
│   ├── main.go               # All endpoints (auth, CRUD, streaming)
│   ├── go.mod
│   ├── go.sum
│   └── .env.example          # Required environment variables
│
├── web/                      # React (Vite) frontend
│   └── src/
│       ├── App.tsx
│       └── GoogleDriveDownload.tsx   # Login, file list, upload, download, delete
│
├── mobile/                   # React Native (Expo) frontend
│   └── app/(tabs)/
│       └── download.tsx      # Same features, native UI
│
└── README.md

API Endpoints

Method Path Query Params Server Memory Description
GET /health — ~0 Health check
GET /auth/login session ~0 Returns Google OAuth2 consent URL
GET /auth/callback code, state ~0 Google redirects here after login
GET /auth/status session ~0 Check if session is authenticated
POST /auth/logout session ~0 Clears session token
GET /files session, pageSize, pageToken, q ~0 List files (paginated)
GET /files/download session, fileId ~32KB Stream download
POST /files/upload session, name, folderId (optional) ~32KB Stream upload (body = raw bytes)
DELETE /files/delete session, fileId ~0 Delete file from Drive

All endpoints require session (except /health and /auth/callback).

The q parameter on /files accepts Google Drive search query syntax, for example:

  • name contains 'report'
  • mimeType = 'application/pdf'
  • 'FOLDER_ID' in parents

Memory & Resource Usage

This is the core advantage — here's exactly what your server uses for each operation:

Operation File Size Server RAM Server Disk Network
Download 10MB 10 MB ~32 KB 0 B 10 MB in + out
Download 2GB 2 GB ~32 KB 0 B 2 GB in + out
Download 50GB 50 GB ~32 KB 0 B 50 GB in + out
Upload 500MB 500 MB ~32 KB 0 B 500 MB in + out
Delete any file any ~0 0 B ~1 KB
List 100 files — ~few KB (JSON) 0 B ~few KB

The 32KB comes from Go's io.Copy internal buffer. This is constant — it does not scale with file size.

Concurrent users: 1,000 simultaneous downloads = 1,000 x 32KB = ~32MB of RAM. That's it. A $5/month VPS can handle this.

What your server stores

  • OAuth tokens in memory (one oauth2.Token struct per active session, ~500 bytes each)
  • The Go binary itself (~15MB)
  • Nothing else. No files. No cache. No temp directory.

Cost Analysis

This approach (Go proxy + Google Drive)

Item Cost
Google Drive storage Free up to 15 GB (personal), or included in Google Workspace ($6-18/user/month)
Google Drive API calls Free — no per-request charges
Server (VPS/Cloud Run) $5-20/month for a small instance
Bandwidth Depends on hosting — many VPS include 1-5TB/month
Total for small project $5-20/month

Traditional approach (your own storage)

Item Cost
AWS S3 storage $0.023/GB/month (100GB = $2.30/month)
S3 egress bandwidth $0.09/GB after 100GB free (1TB = ~$90/month)
Server storage (if local) $0.10-0.30/GB/month on cloud disks
Server RAM for buffering Need enough RAM to buffer uploads/downloads
Backup & replication Additional cost
Total for 1TB traffic $50-150/month+

When this approach wins

  • You already pay for Google Workspace (storage is included)
  • Your files live in Google Drive naturally (documents, spreadsheets, team files)
  • You want zero storage management on your infrastructure
  • Your server budget is minimal (the proxy runs on the smallest VPS possible)
  • You don't need CDN-level latency (files are served from Google's infrastructure)

When traditional storage wins

  • You need sub-50ms latency (CDN edge caching)
  • You have millions of small files accessed frequently (S3 is optimized for this)
  • You need fine-grained access control beyond what Google Drive offers
  • Your files are generated by your backend (not user-uploaded to Drive)

Advantages

  1. Zero server storage — Files never touch your disk. A 500GB Google Drive costs $0 extra if you have Workspace. Storing 500GB on S3 costs ~$11.50/month plus bandwidth.

  2. Constant memory usage — Whether the file is 1KB or 50GB, your server uses ~32KB. This means you can run on the cheapest possible instance.

  3. No file sync — No need to keep your storage in sync with Google Drive. Drive IS the storage. One source of truth.

  4. Google handles durability — Google Drive provides built-in redundancy, versioning, and 99.9% SLA. You get enterprise-grade storage for free.

  5. User-based access — Each user authenticates with their own Google account. They see their own files. No need to build an access control layer.

  6. Native Google Workspace integration — Users can still edit files in Google Docs/Sheets/Slides. Your proxy is an additional access path, not a replacement.

  7. Simple deployment — One Go binary, two environment variables. No database, no cache layer, no storage configuration. Deploy on any $5 VPS, Cloud Run, Fly.io, Railway, or even a Raspberry Pi.

  8. Bandwidth from Google — Download bandwidth comes from Google's infrastructure, not your server's outbound pipe. Google's network is one of the fastest in the world.


Limitations & Trade-offs

  1. Google API rate limits — Google Drive API has a default quota of 20,000 requests per 100 seconds per project. For most applications this is plenty. You can request increases.

  2. Latency — Each request makes a round trip: Client → Your Server → Google → Your Server → Client. This adds ~50-200ms compared to serving files directly from a CDN. For large file downloads, this is negligible. For many small files loaded on a page, consider caching metadata.

  3. No resumable downloads (in this demo) — If a 2GB download fails at 1.5GB, it restarts from the beginning. Production systems should implement HTTP Range requests for resumability.

  4. In-memory token store — This demo stores OAuth tokens in Go's memory. If the server restarts, users must re-authenticate. For production, store tokens in Redis or a database.

  5. Google Drive file size limits — Individual files can be up to 5TB. Upload via the API is limited to 5TB but very large uploads need resumable upload sessions (not implemented in this demo).

  6. Google native formats — Google Docs, Sheets, Slides cannot be downloaded as-is. They must be exported to a format (PDF, XLSX, etc.). This demo skips the download button for these types. You can add export support via the Files.Export() API.

  7. Bandwidth cost is doubled — Data flows through your server, so you pay bandwidth twice: Google → Your Server, then Your Server → Client. On most VPS plans with included bandwidth this doesn't matter. On metered cloud providers (AWS, GCP), this matters at scale.

  8. Single point of failure — If your proxy goes down, clients can't access files. In production, run multiple instances behind a load balancer.


Setup

1. Google Cloud Console

  1. Go to Google Cloud Console
  2. Create a new project (or use an existing one)
  3. Enable the Google Drive API:
    • Go to APIs & Services > Library
    • Search for "Google Drive API"
    • Click Enable
  4. Create OAuth 2.0 credentials:
    • Go to APIs & Services > Credentials
    • Click "Create Credentials" > "OAuth 2.0 Client ID"
    • Application type: Web application
    • Name: anything (e.g., "go-cloud proxy")
    • Authorized redirect URIs: add http://localhost:8080/auth/callback
    • Click Create
    • Copy the Client ID and Client Secret
  5. Configure the OAuth consent screen:
    • Go to APIs & Services > OAuth consent screen
    • User type: Internal (if Google Workspace) or External (for testing)
    • Fill in app name, support email
    • Add scope: https://www.googleapis.com/auth/drive
    • If External: add your email as a test user

2. Backend

Requirements: Go 1.23+

cd backend

# Set environment variables
export GOOGLE_CLIENT_ID="your-client-id.apps.googleusercontent.com"
export GOOGLE_CLIENT_SECRET="your-client-secret"
export GOOGLE_REDIRECT_URL="http://localhost:8080/auth/callback"
export PORT="8080"

# Install dependencies and run
go mod tidy
go run main.go

# Output: Google Drive Proxy listening on :8080

Test health check:

curl http://localhost:8080/health
# ok

3. Web (React + Vite)

Requirements: Node.js 18+

cd web
npm install
npm run dev

# Open http://localhost:5173

What you'll see:

  1. A "Login with Google" button
  2. Click it — opens Google consent screen in a new tab
  3. Approve access — tab closes automatically
  4. File list loads — your Google Drive files appear
  5. Download, Upload, or Delete files from the UI

4. Mobile (React Native + Expo)

Requirements: Node.js 18+, Expo CLI, iOS Simulator or Android Emulator

cd mobile
npm install
npx expo start

Press i for iOS Simulator or a for Android Emulator.

Navigate to the Download tab:

  1. Tap "Login with Google"
  2. In-app browser opens Google consent
  3. Approve access — browser closes, app reloads
  4. File list appears with DL (download) and DEL (delete) buttons

Note for physical devices: Change API_URL in download.tsx to your machine's local IP (e.g., http://192.168.1.100:8080).


Authentication Flow

The authentication uses standard OAuth 2.0 Authorization Code flow:

┌──────────┐     1. /auth/login        ┌──────────┐
│ Frontend  │ ───────────────────────► │ Go Proxy  │
│           │ ◄─── { url: "..." }  ──  │           │
│           │                          │           │
│           │     2. Open URL          │           │
│           │ ──────────────────────►  Google Consent
│           │ ◄─── user approves ───   Screen
│           │                          │           │
│           │     3. /auth/callback    │           │
│  (Google  │ ──── ?code=XXX ────────► │ exchanges │
│  redirects│                          │ code for  │
│  here)    │                          │ token     │
│           │     4. /auth/status      │           │
│           │ ───────────────────────► │           │
│           │ ◄── { authenticated:true}│           │
│           │                          │           │
│           │     5. /files, etc.      │           │
│           │ ───── session=abc ─────► │ uses token│
│           │ ◄── file data ─────────  │ for API   │
└──────────┘                          └──────────┘

Key points:

  • The session ID is generated by the frontend (UUID for web, random string for mobile) and persisted locally (sessionStorage for web, file for mobile)
  • The Go backend maps session ID → OAuth token in memory
  • Tokens include a refresh token (AccessTypeOffline), so they survive access token expiration
  • The frontend never sees Google credentials — it only knows its own session ID

Production Considerations

Before deploying to production, consider these improvements:

Token Storage

Replace the in-memory tokenStore with persistent storage:

// Instead of sync.Map in memory, use:
// - Redis (fast, good for multiple instances)
// - PostgreSQL/MySQL (durable, good for single instance)
// - Encrypted file on disk (simplest, single instance only)

CORS

Replace the * wildcard with your actual frontend domain:

w.Header().Set("Access-Control-Allow-Origin", "https://your-app.com")

HTTPS

Always use HTTPS in production. Put the proxy behind a reverse proxy (nginx, Caddy) or deploy on a platform that provides TLS (Cloud Run, Fly.io, Railway).

Rate Limiting

Add per-session rate limiting to prevent abuse:

// Use golang.org/x/time/rate
limiter := rate.NewLimiter(10, 20) // 10 requests/sec, burst 20

Resumable Downloads

For large files, implement HTTP Range header support so interrupted downloads can resume instead of restarting.

Resumable Uploads

For large files (>5MB), use Google's resumable upload protocol instead of simple upload. This allows retry on failure without re-uploading from the start.

Session Expiry

Add a TTL to sessions so abandoned sessions don't accumulate tokens in memory:

// Clean up sessions older than 24 hours
go func() {
    for range time.Tick(1 * time.Hour) {
        store.CleanExpired(24 * time.Hour)
    }
}()

Logging & Monitoring

Add structured logging (zerolog, slog) and metrics (Prometheus) to track:

  • Active sessions
  • Downloads/uploads per minute
  • Bytes transferred
  • Error rates
  • Google API quota usage

License

MIT

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages