🔍 GitHub Actions Workflow Comparison: Two Valid Approaches

Options Trading System

name: Deploy Options Trading Systems to AWS Lambda
on:
  push:
    branches:
      - main
  workflow_dispatch:
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'
      - name: Install AWS SAM CLI
        run: |
          pip install aws-sam-cli
      # Deploy live trading system
      - name: Build and deploy live trading
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: options-trading-system
          IMAGE_TAG: ${{ github.sha }}
        run: |
          cd Enter_Mon_Hold_Till_Fri
          docker buildx build \
            --cache-from type=registry,ref=$ECR_REGISTRY/$ECR_REPOSITORY:cache-live \
            --cache-to type=registry,ref=$ECR_REGISTRY/$ECR_REPOSITORY:cache-live,mode=max \
            -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
            --push \
            --provenance=false .
          sam deploy \
            --no-confirm-changeset \
            --no-fail-on-empty-changeset \
            --stack-name options-trading-system \
            --capabilities CAPABILITY_IAM \
            --region us-east-1 \
            --resolve-s3 \
            --resolve-image-repos \
            --parameter-overrides ImageUri=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
      - name: Clean up old ECR images
        env:
          ECR_REPOSITORY: options-trading-system
        run: |
          echo "🧹 Cleaning up old Docker images from $ECR_REPOSITORY..."
          OLD_IMAGES=$(aws ecr describe-images \
            --repository-name $ECR_REPOSITORY \
            --region us-east-1 \
            --query 'sort_by(imageDetails,&imagePushedAt)[:-3].imageDigest' \
            --output text)
          if [ -n "$OLD_IMAGES" ]; then
            echo "$OLD_IMAGES" | tr '\t' '\n' | while read digest; do
              if [ -n "$digest" ]; then
                echo "  ❌ Deleting image: $digest"
                aws ecr batch-delete-image \
                  --repository-name $ECR_REPOSITORY \
                  --image-ids imageDigest=$digest \
                  --region us-east-1 \
                  --no-cli-pager || true
              fi
            done
          else
            echo "  ✅ No old images to delete (keeping 3 most recent)"
          fi
          REMAINING=$(aws ecr describe-images \
            --repository-name $ECR_REPOSITORY \
            --region us-east-1 \
            --query 'length(imageDetails)' \
            --output text)
          echo "🎉 Cleanup complete! $ECR_REPOSITORY now has $REMAINING images"

Sector Rotation System

name: Deploy Sector Rotation System to AWS Lambda
on:
  push:
    branches:
      - main
  workflow_dispatch:
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
      - name: Build Docker image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: etf-trading-system
          IMAGE_TAG: ${{ github.sha }}
        run: |
          cd live_trading
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'
      - name: Install AWS SAM CLI
        run: |
          pip install aws-sam-cli
      - name: Deploy with SAM
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: etf-trading-system
          IMAGE_TAG: ${{ github.sha }}
        run: |
          sam deploy \
            --no-confirm-changeset \
            --no-fail-on-empty-changeset \
            --stack-name etf-trading-system \
            --capabilities CAPABILITY_IAM \
            --region us-east-1 \
            --resolve-s3 \
            --resolve-image-repos \
            --parameter-overrides ImageUri=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
      - name: Clean up old ECR images
        env:
          ECR_REPOSITORY: etf-trading-system
        run: |
          echo "🧹 Cleaning up old Docker images from $ECR_REPOSITORY..."
          OLD_IMAGES=$(aws ecr describe-images \
            --repository-name $ECR_REPOSITORY \
            --region us-east-1 \
            --query 'sort_by(imageDetails,&imagePushedAt)[:-3].imageDigest' \
            --output text)
          if [ -n "$OLD_IMAGES" ]; then
            echo "$OLD_IMAGES" | tr '\t' '\n' | while read digest; do
              if [ -n "$digest" ]; then
                echo "  ❌ Deleting image: $digest"
                aws ecr batch-delete-image \
                  --repository-name $ECR_REPOSITORY \
                  --image-ids imageDigest=$digest \
                  --region us-east-1 \
                  --no-cli-pager || true
              fi
            done
          else
            echo "  ✅ No old images to delete (keeping 3 most recent)"
          fi
          REMAINING=$(aws ecr describe-images \
            --repository-name $ECR_REPOSITORY \
            --region us-east-1 \
            --query 'length(imageDetails)' \
            --output text)
          echo "🎉 Cleanup complete! $ECR_REPOSITORY now has $REMAINING images"

📊 Key Structural Differences

✓ Both Workflows Work Correctly

These are two valid organizational patterns with different tradeoffs:

Options Trading: Self-contained subdirectory with template + code, combined build/deploy step
Sector Rotation: Template at repo root, code in subdirectory, separate build/deploy steps

Aspect	Options Trading	Sector Rotation
Directory Structure	Self-contained approach: Enter_Mon_Hold_Till_Fri/ ├── template.yaml ├── Dockerfile └── lambda_function.py	Separated approach: ├── template.yaml └── live_trading/ ├── Dockerfile └── lambda_function.py
Step Organization	1. Checkout 2. AWS Creds 3. ECR Login 4. Docker Buildx Setup 5. Python Setup 6. SAM Install 7. Build + Deploy (COMBINED) 8. Cleanup	1. Checkout 2. AWS Creds 3. ECR Login 4. Docker Build (SEPARATE) 5. Python Setup 6. SAM Install 7. SAM Deploy (SEPARATE) 8. Cleanup
Docker Tool	docker buildx (advanced)	docker build (standard)
Build Directory	Enter_Mon_Hold_Till_Fri/	live_trading/
SAM Deploy Directory	✓ Enter_Mon_Hold_Till_Fri/ (finds template.yaml there)	✓ Repo root (finds template.yaml there)
Docker Push	Integrated with --push flag	Separate docker push command
Registry Caching	✓ Uses --cache-from/--cache-to	✗ No caching

🎯 Pattern 1: Self-Contained Subdirectory (Options Trading)

✓ Self-Contained Approach

Philosophy: Everything for this system lives in one subdirectory

cd Enter_Mon_Hold_Till_Fri

→

docker buildx build

→

sam deploy (finds template.yaml here)

Enter_Mon_Hold_Till_Fri/
├── template.yaml              # Template with the code
├── Dockerfile                 
├── lambda_function.py
├── requirements.txt
└── (data files, models, etc)

Workflow step:
- name: Build and deploy live trading
  run: |
    cd Enter_Mon_Hold_Till_Fri         # Enter the self-contained directory
    docker buildx build ...             # Build
    sam deploy ...                      # Deploy (finds template.yaml here)
                

Benefits:

Everything in one place - easy to move/copy the entire system
Clear module boundary - this directory IS the trading system
Single step deployment - build and deploy together
Uses advanced Docker buildx with layer caching

Tradeoffs:

Template and code tightly coupled in same directory
If you have multiple systems, each needs its own subdirectory with template

🎯 Pattern 2: Root Template with Code Subdirectory (Sector Rotation)

✓ Separated Approach

Philosophy: Infrastructure config (template) at root, application code in subdirectory

Step 1: cd live_trading

→

docker build

⬇ Step ends - directory resets to repo root ⬇

Step 2: sam deploy (finds template.yaml at root)

repo-root/
├── template.yaml              # Infrastructure at top level
└── live_trading/
    ├── Dockerfile             
    ├── lambda_function.py
    ├── requirements.txt
    └── (trading logic)

Workflow steps:
- name: Build Docker image
  run: |
    cd live_trading                    # Build code in subdirectory
    docker build ...
    # Step ends → back to root

- name: Deploy with SAM
  run: |
    sam deploy ...                     # Deploy from root (finds template.yaml)
                

Benefits:

Clean separation: infrastructure vs application code
Template visible at top level - easy to find and modify
Separate build/deploy steps - clearer logging, easier debugging
Common pattern in many repos

Tradeoffs:

Code and infrastructure split across directories
No Docker buildx caching (uses standard docker build)
Two steps instead of one

🔧 Docker Build Comparison

Feature	Options Trading	Sector Rotation
Build Command	`docker buildx build`	`docker build`
Cache Strategy	`--cache-from type=registry` `--cache-to type=registry,mode=max` Benefit: Faster builds by reusing layers	None Impact: Rebuilds from scratch each time
Push Method	Integrated: `--push` flag Benefit: Single command	Separate: `docker push` Impact: Two commands needed
Provenance	`--provenance=false`	Default (not specified)
Requires Buildx Setup	✓ Yes (separate step)	✗ No (uses standard Docker)

💡 GitHub Actions Step Behavior

Important: Each Step Resets Working Directory

In GitHub Actions, every step starts with a fresh shell session at the repository root. Any cd commands only affect that specific step.

- name: Step 1
  run: |
    cd some/subdirectory
    echo "I'm in $(pwd)"
    # Step ends → directory change is forgotten

- name: Step 2
  run: |
    echo "I'm in $(pwd)"    # Back at repo root!
                

This is why Sector Rotation works: the Docker build step changes directory, but the SAM deploy step starts fresh at the repo root.

🤔 Which Pattern Should You Use?

Consideration	Self-Contained (Options)	Root Template (Sector)
Multiple systems in repo	✓ Each system is independent directory	⚠️ Need multiple templates at root or complex template
Easy to relocate	✓ Copy entire directory	⚠️ Need to move both root template and subdirectory
Infrastructure visibility	⚠️ Template buried in subdirectory	✓ Template obvious at repo root
Build performance	✓ Docker buildx with layer caching	Standard build, no caching
Step clarity	Single combined step	✓ Separate steps, clearer logs
Common convention	Less common (but valid)	✓ More typical repo structure

🔄 Converting Between Patterns

From Self-Contained → Root Template

# Move template to root
mv Enter_Mon_Hold_Till_Fri/template.yaml ./

# Split workflow steps
- name: Build Docker image
  run: |
    cd Enter_Mon_Hold_Till_Fri
    docker build ...

- name: Deploy with SAM
  run: |
    sam deploy ...
            

From Root Template → Self-Contained

# Move template into subdirectory
mv template.yaml live_trading/

# Combine workflow steps
- name: Build and deploy
  run: |
    cd live_trading
    docker build ...
    sam deploy ...
            

📝 Summary

Pattern	Status	Key Characteristic	Best For
Options Trading (Self-Contained)	✓ WORKS	Everything in subdirectory, single combined step, buildx caching	Multi-system repos where each system should be independent and portable
Sector Rotation (Root Template)	✓ WORKS	Template at root, code in subdirectory, separate build/deploy steps	Single-system repos following conventional project structure

💡 Key Takeaways

Both patterns are valid and work correctly - choose based on your repo structure needs
Options Trading uses a self-contained approach where template.yaml lives WITH the code
Sector Rotation uses a separated approach where template.yaml lives at repo root
The key difference is organizational philosophy, not correctness
GitHub Actions step behavior: Each step starts fresh at repo root, but you can cd anywhere within a step

🔍 Technical Deep Dive: Docker Build Differences

Why Options Trading Uses Docker Buildx

# Standard docker build (Sector Rotation)
docker build -t image:tag .
docker push image:tag

# Advanced docker buildx (Options Trading)  
docker buildx build \
  --cache-from type=registry,ref=image:cache \
  --cache-to type=registry,ref=image:cache,mode=max \
  -t image:tag \
  --push \
  --provenance=false .
            

Buildx advantages:

Layer caching: Reuses unchanged layers from previous builds → faster CI/CD
Integrated push: Build and push in one command
Registry-backed cache: Cache survives across CI runners
Advanced features: Multi-platform builds, attestations, etc.

When is buildx overkill?

Small codebases with fast builds (like Sector Rotation)
Infrequent deployments where build time doesn't matter
Simple Docker images without complex dependency chains