Skip to content

The GenericSuite for Python

GenericSuite (backend version) is a versatile backend solution, designed to provide a comprehensive suite of features for Python APIs. It supports various frameworks including FastAPI, Flask and Chalice, making it adaptable to a range of projects. This repository contains the backend logic, utilities, and configurations necessary to build and deploy scalable and maintainable applications.

Features

  • Framework Agnostic: Supports FastAPI, Flask, and Chalice frameworks.
  • Database Support: Includes abstracted database operations for both MongoDB and DynamoDB, offering flexibility in choosing the database.
  • Authentication: Implements JWT-based authentication, providing secure access to endpoints.
  • Dynamic Endpoint Creation: Allows for defining endpoints dynamically through JSON configurations.
  • Utilities: A collection of utilities for tasks such as sending emails, parsing multipart data, handling passwords, and more.
  • Billing Utilities: Tools for managing billing plans and user subscriptions.
  • Menu Options: Functionality to manage and retrieve authorized menu options based on user roles.

Pre-requisites

AWS account and credentials

If you plan to deploy the App in the AWS Cloud:

Getting Started

To get started with GenericSuite, follow these steps:

Initiate your project

To create the project directory for the App's backend API. E.g. exampleapp_backend, instrctions will depend on the dependency management of your preference:

# Pip
mkdir -p exampleapp_backend/exampleapp_backend
cd exampleapp_backend
python3 -m venv venv
source venv/bin/activate

# Pipenv
# https://docs.pipenv.org/basics/
mkdir -p exampleapp_backend/exampleapp_backend
cd exampleapp_backend
pipenv install

# Poetry
# https://python-poetry.org/docs/basic-usage/
poetry start exampleapp_backend
cd exampleapp_backend

Installation

To use GenericSuite in your project, install it with the following command(s):

From PyPi

Pip

pip install genericsuite

Pipenv

pipenv install genericsuite

Poetry

poetry add genericsuite

NOTE: in the following instructions we'll only show pip install ....
If you'll use Pipenv, change it with pipenv install ....
If you'll use Poetry, change it with poetry add ....

From Git or Local Directory

Check this documentation to install from a Git repository/branch or a Local Directory.

Framework installation

Install the desired framework: FastAPI, Flask or Chalice:

FastAPI

pip install fastapi fastapi-cors "uvicorn[standard]" python-multipart mangum

Flask

pip install Flask

Chalice

pip install chalice

For more information:

Test dependencies

To execute the unit and integration test, install pytest and coverage:

pip install pytest coverage

Development scripts installation

The GenericSuite backend development scripts contains utilities to build and deploy APIs made by The GenericSuite.

npm install -D genericsuite-be-scripts

Available options

  1. Select Your Framework: Depending on your project, choose between FastAPI, Flask or Chalice.
  2. Select Your Database of choice: Implement database operations using the provided abstracted functions for MongoDB and DynamoDB.
  3. Included Authentication: Your endpoints will be secured with JWT-based authentication.
  4. Define Endpoints: Utilize the dynamic endpoint creation feature by defining your endpoints in a JSON configuration file. Visit the Generic Suite Configuration Guide for more information.
  5. Define Menu Options: Utilize the dynamic menu creation feature by defining your muenu and option access security in a JSON configuration file. Visit the Generic Suite Configuration Guide for guidance.
  6. Define Table structures: Utilize the dynamic table creation feature by defining your CRUD editors in JSON configuration files. Visit the Generic Suite Configuration Guide for sample code and files.

Configuration

Configure your application by setting up the necessary environment variables. Refer to the .env.example and config.py files for the available options.

  • Aplicacion name
APP_NAME=ExampleApp
  • Aplicacion domain
APP_DOMAIN_NAME=exampleapp.com
  • Application default language
DEFAULT_LANG=en
  • Stage and Debug flag
# DEV
# Application debug (0,1)
APP_DEBUG=1
# Application environment: dev, qa, staging, prod
# APP_STAGE=dev

# QA
APP_DEBUG=1
# APP_STAGE=qa

# PROD
APP_DEBUG=0
# APP_STAGE=prod
  • Application secret keys
# Application secret key (to be used in password encryption)
APP_SECRET_KEY=xxxx

# Storage seed: to be used in storage URL encryption -e.g. AWS S3-
# Generate a new one with: `make generate_seed`
STORAGE_URL_SEED="yyyy"
  • Application super administrator Email
APP_SUPERADMIN_EMAIL=xxxx

# DEV: docker
APP_DB_ENGINE_DEV=DYNAMO_DB
DYNAMDB_PREFIX_DEV=
APP_DB_URI_DEV=http://127.0.0.1:8000

# QA: AWS DynamoDB
APP_DB_ENGINE_QA=DYNAMO_DB
DYNAMDB_PREFIX_QA=
APP_DB_URI_QA=

# PROD: AWS DynamoDB
APP_DB_ENGINE_PROD=DYNAMO_DB
DYNAMDB_PREFIX_PROD=
APP_DB_URI_PROD=

# # DEMO: AWS DynamoDB
# APP_DB_ENGINE_DEMO=DYNAMO_DB
# DYNAMDB_PREFIX_DEMO=
# APP_DB_URI_DEMO=

NOTE: set DYNAMDB_PREFIX_* empty and it'll be defaulted to <APP_NAME_LOWERCASE>_<STAGE>_.

  1. For MongoDB
    https://www.mongodb.com/
# DEV: Docker container
APP_DB_ENGINE_DEV=MONGO_DB
APP_DB_NAME_DEV=mongo
APP_DB_URI_DEV=mongodb://root:example@app.exampleapp.local:27017/

# QA: MongoDB Atlas
APP_DB_ENGINE_QA=MONGO_DB
APP_DB_NAME_QA=xxxx
APP_DB_URI_QA=mongodb+srv://<user>:<password>@<cluster>.mongodb.net

# Staging: MongoDB Atlas
APP_DB_ENGINE_STAGING=MONGO_DB
APP_DB_NAME_STAGING=xxxx
APP_DB_URI_STAGING=mongodb+srv://<user>:<password>@<cluster>.mongodb.net

# PROD: MongoDB Atlas
APP_DB_ENGINE_PROD=MONGO_DB
APP_DB_NAME_PROD=xxxx
APP_DB_URI_PROD=mongodb+srv://<user>:<password>@<cluster>.mongodb.net

# DEMO: MongoDB Atlas
APP_DB_ENGINE_DEMO=MONGO_DB
APP_DB_NAME_DEMO=xxxx
APP_DB_URI_DEMO=mongodb+srv://<user>:<password>@<cluster>.mongodb.net
  • CORS Origin
# DEV
APP_CORS_ORIGIN_DEV=*

# QA
APP_CORS_ORIGIN_QA=*
APP_CORS_ORIGIN_QA_CLOUD=https://app-qa.exampleapp.com
APP_CORS_ORIGIN_QA_LOCAL=http://localhost:3000

# Staging
APP_CORS_ORIGIN_STAGING=https://app-qa.exampleapp.com

# PROD
APP_CORS_ORIGIN_PROD=https://app.exampleapp.com

# DEMO
APP_CORS_ORIGIN_DEMO=https://app-demo.exampleapp.com
  • Current framework options: chalice, flask, fastapi
CURRENT_FRAMEWORK=chalice
  • JSON configuration files location and git URL
GIT_SUBMODULE_LOCAL_PATH=lib/config_dbdef
GIT_SUBMODULE_URL=git://github.com/username/exampleapp_configs.git
  • Frontend application path (to copy version file during big lambdas deployment)
FRONTEND_PATH=../exampleapp_frontend
  • Local python version
PYTHON_VERSION=3.11.5
# PYTHON_VERSION=3.10.12
# PYTHON_VERSION=3.9.17
  • IAAS Cloud provider
# IAAS Cloud provider
# Available options: `aws`, `gcp`, `azure`
CLOUD_PROVIDER=aws
  • Enable/disable Cloud Provider secrets
# Enable/disable Cloud Provider secrets (instead of environment variables).
# Available options: `1` to enable, `0` to disable. Defaults to: 1
# GET_SECRETS_ENABLED=0
#
# Fine grained Cloud Provider secrets management:
#
# Enable/disable Cloud Provider envvars.
# Available options: `1` to enable, `0` to disable. Defaults to: 1
# Set to "0" in local development environment so envvars like APP_CORS_ORIGIN can be
# set by the scripts and .env file and access QA resources from DEV.
# GET_SECRETS_ENVVARS=0
#
# Enable/disable Cloud Provider critical secrets.
# Available options: `1` to enable, `0` to disable. Defaults to: 1
# Set to "0" in local development environment so envvars like APP_DB_URI can be
# set by the scripts and .env file and access QA resources from DEV.
# GET_SECRETS_CRITICAL=0

# AWS S3 bucket name (used by set_fe_cloudfront_domain.sh to set the CloudFront domain name in the frontend for the CORS config)
AWS_S3_BUCKET_NAME_FE=exampleapp-frontend-website-[STAGE]

# Region for this App all AWS services
AWS_REGION=aws-region

# AWS base name for Lambda Functions, API Gateway, EC2, ELB, etc.
AWS_LAMBDA_FUNCTION_NAME=exampleapp-backend

# AWS Lambda function role:
# These variables are used only if deploy without AWS SAM (deploy_without_sam) in big_lambdas_manager.sh. SAM generates this role automatically
AWS_LAMBDA_FUNCTION_ROLE_QA=exampleapp-api_handler-role-qa
AWS_LAMBDA_FUNCTION_ROLE_STAGING=exampleapp-api_handler-role-staging
AWS_LAMBDA_FUNCTION_ROLE_DEMO=exampleapp-api_handler-role-demo
AWS_LAMBDA_FUNCTION_ROLE_PROD=exampleapp-api_handler-role-prod

# AWS SSL certificate ARN (used by big_lambdas_manager.sh)
AWS_SSL_CERTIFICATE_ARN=arn:aws:acm:AWS-REGION:AWS-ACCOUNT:certificate/AWS-CERTIFICATE-UUID

# AWS Deployment type
# Available options: `lambda`, `fargate`, `ec2`. Defaults to: lambda
AWS_DEPLOYMENT_TYPE=lambda
  • Assests URL masking external hostname
# For features like AI Vision, only in development environment, used by the storage URL encryption.
# E.g. app-dev.exampleapp.com
# Leave blank to use the same URL stored, for example in the AI Assistant conversarions.
URL_MASK_EXTERNAL_HOSTNAME=
  • SMTP Mail configuration
SMTP_SERVER=smtp_server
SMTP_PORT=smtp_port
SMTP_USER=smtp_user
SMTP_PASSWORD=smtp_password
SMTP_DEFAULT_SENDER=sender_email
  • Docker configuration
DOCKER_ACCOUNT=docker_account_username
  • Local development environment run configuration
# Options are: uvicorn, gunicorn, chalice, chalice_docker
# Chalice case: "chalice" to use http (running without docker) or "chalice_docker" to use https (with docker)
# http:
# RUN_METHOD="chalice"
# https:
RUN_METHOD="chalice_docker"
  • Tests configuration
# Testing endpoint
TEST_APP_URL=http://app.exampleapp.local:5001
  • Run methods and framework App directory and entry point
#
# Default App main code directory
# for Chalice:
# https://aws.github.io/chalice/topics/packaging.html
# APP_DIR='.'
# for FastAPI:
# https://fastapi.tiangolo.com/tutorial/bigger-applications/?h=directory+structure#an-example-file-structure
# APP_DIR='app'
# for Flask:
# https://flask.palletsprojects.com/en/2.3.x/tutorial/layout/
# APP_DIR='flaskr'
#
# Default App entry point code file
# for Chalice:
# https://aws.github.io/chalice/topics/packaging.html
# APP_MAIN_FILE='app'
# for FastAPI:
# https://fastapi.tiangolo.com/tutorial/bigger-applications/?h=directory+structure#an-example-file-structure
# APP_MAIN_FILE='main'
# for Flask:
# https://flask.palletsprojects.com/en/2.3.x/tutorial/factory/
# APP_MAIN_FILE='__init__'
#
```env

* App local ports
```env
# Local frontend port (defaults to 3000)
FRONTEND_LOCAL_PORT=3000
#
# Local backend API port (defaults to 5001)
BACKEND_LOCAL_PORT=5001
  • Disable local services
    (useful when running the local dev environment on the road, offline, over a smartphone internet connection)
# Disable local DNS server startup during app run
LOCAL_DNS_DISABLED=1
# Disable bridge proxy startup during app run
BRIDGE_PROXY_DISABLED=1
  • Flask configuration
FLASK_APP=__init__.py

# Localstack configuration
# LOCALSTACK_AUTH_TOKEN=""
# (Set LOCALSTACK_AUTH_TOKEN empty when working offline, and assign the Auth Token to make services like EC2 to work correctly)

App structure

Suggested directory structure by framework:

This is a suggested App development repository structure for a FastAPI project:

.
├── app
│   ├── __init__.py
│   ├── main.py
│   ├── dependencies.py
│   └── routers
│   │   ├── __init__.py
│   │   ├── items.py
│   │   └── users.py
│   └── internal
│       ├── __init__.py
│       └── admin.py
├── logs
│   └── .gitignore
├── .env
├── .env.example
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
├── package-lock.json
├── package.json
├── requirements.txt
├── tests
│   ├── .env.for_test
│   ├── __init__.py
│   ├── assets
│   ├── conftest.py
│   └── pytest.ini
└── version.txt

This is a suggested App development repository structure for a Flask project:

.
├── flaskr/
│   ├── __init__.py
│   ├── items.py
│   ├── users.py
│   ├── admin.py
│   └── index.py
├── logs
│   └── .gitignore
├── package-lock.json
├── package.json
├── requirements.txt
├── tests
│   ├── .env.for_test
│   ├── __init__.py
│   ├── assets
│   ├── conftest.py
│   └── pytest.ini
├── .env
├── .env.example
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── Pipfile
├── Pipfile.lock
├── README.md
└── version.txt

This is a suggested App development repository structure for a Chalice project:

.
├── .chalice
│   ├── config-example.json
│   ├── config.json
│   ├── deployed
│   │   ├── dev.json
│   │   ├── qa.json
│   │   └── prod.json
│   ├── deployment
│   │   ├── deployment.zip
│   │   └── sam.json
│   ├── deployments
│   ├── dynamodb_cf_template.yaml
│   └── policy-qa.json
├── chalicelib
│   └── endpoints
│       ├── items.py
│       ├── users.py
│       ├── admin.py
│       └── __init__.py
├── lib
│   ├── .gitignore
│   ├── config
│   │   ├── __init__.py
│   │   └── config.py
│   ├── config_dbdef
│   │   ├── .gitignore
│   │   ├── CHANGELOG.md
│   │   ├── README.md
│   │   ├── backend
│   │   └── frontend
│   └── models
│       ├── __init__.py
│       ├── ai_chatbot
│       │   ├── __init__.py
│       │   └── ai_gpt_fn_index.py
│       ├── external_apis
│       │   └── __init__.py
│       └── utilities
├── logs
│   └── .gitignore
├── tests
│   ├── .env.for_test
│   ├── __init__.py
│   ├── assets
│   ├── conftest.py
│   └── pytest.ini
├── .env
├── .env.example
├── .gitignore
├── app.py
├── CHANGELOG.md
├── LICENSE
├── Makefile
├── package-lock.json
├── package.json
├── Pipfile
├── Pipfile.lock
├── README.md
├── requirements.txt
└── version.txt

Code examples and JSON configuration files

The main menu, API endpoints and CRUD editor configurations are defined in the JSON configuration files.

You can find examples about configurations and how to code an App in the GenericSuite App Creation and Configuration guide.

Usage

Check the The GenericSuite backend development scripts for more details.

License

This project is licensed under the ISC License - see the LICENSE file for details.

Credits

This project is developed and maintained by Carlos J. Ramirez. For more information or to contribute to the project, visit GenericSuite on GitHub.

Happy Coding!