Background
I've been working with Elixir Phoenix since I think 1.4 and I enjoy the functional programming, the lightweight VMS, the ability to scale, and the relative size and enthusiasm of the ecosystem of vendors and community.
This website is hosted on a Patreon or Substack style subscription hosting clone I built with Phoenix with some Classic Views and also LiveViews.
I wanted to jump ahead and use the Ash Phoenix Framework from Alembic and Zach Daniel.
So in this project, which I literally just started, I am using the Ash API JSON Wrapper to consume the Ecommerce API OpenAPI (formerly Swagger) spec to generate Ash Resources, from which I will then derive JSON, Postgres, GraphQL, Phoenix LiveView and interface into a Vercel React.JS app called Commerce.
I built a commercial edge hosting pre-sales app using the Vercel Next.JS Commerce app in a previous life and I really like its functionality and performance. Next.JS hits the sweat spot in terms of performance, speed, cacheability and maturity in the ecosystem.
Backend
store
Store
- Shopify inspired Devekko Store
- Ash Phoenix
- Pgvector
- Ollama
- GPU enabled
Frontend
commerce
Commerce
- Forked Next.JS React Commerce app via Vercel
Infra
- Docker Compose
- Podman
- Debian
Docker Compose
I build out a Docker Compose file with GPU enabled containers for NVIDIA. I also pull in Ollama, again running on GPU. Pgvector is used and will vectorize content using Ollama for search and generating content. The Ash app imports Ecommerce API JSON spec and generates Ash Resources from which the Ash Phoenix app will be generated.
Docker Compose GPU Stack
compose.yml
networks:
internet: {}
data: {}
services:
cuda:
image: nvidia/cuda:12.3.1-base-ubuntu20.04
command: nvidia-smi
deploy:
resources:
reservations:
devices:
- capabilities: ["utility"]
count: all
ollama:
volumes:
- ./ollama/ollama:/root/.ollama
container_name: ollama
pull_policy: always
tty: true
restart: unless-stopped
image: ollama/ollama:latest
ports:
- 7869:11434
environment:
- OLLAMA_KEEP_ALIVE=24h
networks:
- data
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
store_ash_dev:
build:
context: ./store
dockerfile: ./gpu.Dockerfile
container_name: store
develop:
watch:
- action: sync
path: ./store
target: /app
# ignore:
# - deps/
# - action: rebuild
# path: package.json
deploy:
resources:
reservations:
devices:
- capabilities: ["utility"]
count: all
# volumes:
# - type: bind
# source: ./folkbot
# target: /app
# - docker-composer-elixir-mix:/root/.mix
# - hex:/root/.hex
networks:
- internet
- data
depends_on:
- store_pgvector_dev
ports:
- "4799:4004"
environment:
- NVIDIA_VISIBLE_DEVICES=all
- DATABASE_URL=ecto://admin:redacted@pgvector/folkbot_prod
- SECRET_KEY_BASE="redacted
# command: sleep infinity
command:
- /app/gpu.sh
store_pgvector_dev:
image: pgvector/pgvector:pg17
container_name: store_pgvector_dev
ports:
- 5799:5432
networks:
- data
restart: always
environment:
- POSTGRES_DB=store_pgvector_dev
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=cohort9theory+Defy
volumes:
- ./pgvector-init:/docker-entrypoint-initdb.d/
- postgres_data:/var/lib/postgresql
#- ./folkbot_prod.sql:/docker-entrypoint-initdb.d/folkbot_prod.sql
# configs:
# - source: pgvector-init/folkbot_prod.sql
# target: /docker-entrypoint-initdb.d/init.sql
volumes:
postgres_data:
Ecommerce API OpenAPI Spec
Ensure interoperability and data transfer between ecommerce platforms using the Ecommerce API OpenAPI spec from APIDeck. The spec when references are resolved is large, 30k lines, its a detailed formal spec and JSON is notoriously verbose. https://developers.apideck.com/apis/ecommerce/reference
List Orders
- get/ecommerce/orders
Get Order
- get/ecommerce/orders/{id}
List Products
- get/ecommerce/products
Get Product
- get/ecommerce/products/{id}
List Customers
- get/ecommerce/customers
Get Customer
- get/ecommerce/customers/{id}
Get Store
- get/ecommerce/store
JSON API Wrapper
Mix task used to spec resources and map back to the Ecommerce JSON
defmodule API.Gen.Ecommerce do
# require IEx
require Ash.Query
# @moduletag :oapi_petstore
use Ash.Domain,
validate_config_inclusion?: false
@json "test/ecommerceapi/newecommerceapi.json" |> File.read!() |> Jason.decode!()
defmodule Domain do
use Ash.Domain,
validate_config_inclusion?: false
resources do
allow_unregistered? true
end
end
@config [
tesla: TestingTesla,
endpoint: "https://developers.apideck.com",
resources: [
# schemas
EcommerceApiAddress:
[
path: "__schema__",
object_type: "components.schemas.Address",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiDepartment:
[
path: "__schema__",
object_type: "components.schemas.Department",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
# EcommerceApiDivision:
# [
# path: "__schema__",
# object_type: "components.schemas.Division",
# primary_key: "",
# # entity_path: "",
# fields: [
# id: [
# filter_handler: {:place_in_csv_list, ["id"]}
# ]
# ]
# ],
EcommerceApiEcommerceAddress:
[
path: "__schema__",
object_type: "components.schemas.EcommerceAddress",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceCustomer:
[
path: "__schema__",
object_type: "components.schemas.EcommerceCustomer",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceDiscount:
[
path: "__schema__",
object_type: "components.schemas.EcommerceDiscount",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceOrderLineItem:
[
path: "__schema__",
object_type: "components.schemas.EcommerceOrderLineItem",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiTrackingItem:
[
path: "__schema__",
object_type: "components.schemas.TrackingItem",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiWebsite:
[
path: "__schema__",
object_type: "components.schemas.Website",
primary_key: "",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
# paths
EcommerceApiEcommerceStore:
[
path: "/ecommerce/store",
object_type: "components.schemas.EcommerceStore",
primary_key: "id",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceOrder:
[
path: "/ecommerce/orders/{id}",
object_type: "components.schemas.EcommerceOrder",
primary_key: "id",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceCustomer:
[
path: "/ecommerce/customers/{id}",
object_type: "components.schemas.EcommerceCustomer",
primary_key: "id",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
],
EcommerceApiEcommerceProduct:
[
path: "/ecommerce/products/{id}",
object_type: "components.schemas.EcommerceProduct",
primary_key: "id",
# entity_path: "",
fields: [
id: [
filter_handler: {:place_in_csv_list, ["id"]}
]
]
]
]
]
def json do
json_schema_resolved = @json_schema.schema
json_resolved = json_schema_resolved |> Jason.encode!()
# IO.puts(json_schema_resolved |> Jason.encode!())
File.write!("priv/generated/ecommerce_api_schema.json", json_resolved)
end
def debug do
json_schema_resolved = @json
IO.inspect(json_schema_resolved)
# File.write!("priv/generated/ecommerce_api_schema.json", json_resolved)
end
def debugcode do
@json
# |> IO.inspect()
# IEx.pry()
|> AshJsonApiWrapper.OpenApi.ResourceGenerator.generate(Domain, @config)
# |> IO.inspect()
|> Enum.map(fn {resource, code} ->
# IO.inspect(resource)
# IO.inspect(code)
# IO.inspect(Code.eval_string(code))
# Code.eval_string(code)
resource_down = to_string(resource) |> Macro.underscore()
IO.inspect(resource_down)
File.write!("priv/generated/debug_#{resource_down}.ex", code)
end)
end
def code do
@json
|> AshJsonApiWrapper.OpenApi.ResourceGenerator.generate(Domain, @config)
|> Enum.map(fn {resource, code} ->
Code.eval_string(code)
resource_down = to_string(resource) |> Macro.underscore()
File.write!("priv/generated/#{resource_down}.ex", code)
end)
end
end
JSON API Resource Generator
The Ash API JSON Resource Generator is forked, and I am experimenting with expanding the intent, features and options, currently looks like:
defmodule AshJsonApiWrapper.OpenApi.ResourceGenerator do
@moduledoc "Generates resources from an open api specification"
def generate(json, domain, main_config) do
main_config[:resources]
|> Enum.map(fn {resource, config} ->
endpoints =
json
|> operations(config)
|> Enum.map_join("\n\n", fn {path, _method, operation} ->
entity_path =
if config[:entity_path] do
"entity_path \"#{config[:entity_path]}\""
end
"""
endpoint :#{operation_id(operation)} do
path "#{path}"
#{entity_path}
end
"""
end)
actions =
json
|> operations(config)
|> Enum.map_join("\n\n", fn
{_path, "get", config} ->
"""
read :#{operation_id(config)}
"""
{_path, "post", config} ->
"""
create :#{operation_id(config)}
"""
end)
fields =
config[:fields]
|> Enum.map_join("\n\n", fn {name, field_config} ->
filter_handler =
if field_config[:filter_handler] do
"filter_handler #{inspect(field_config[:filter_handler])}"
end
"""
field #{inspect(name)} do
#{filter_handler}
end
"""
end)
|> case do
"" ->
""
other ->
"""
fields do
#{other}
end
"""
end
{:ok, [object]} =
json
|> ExJSONPath.eval(config[:object_type])
attributes =
object
|> Map.get("properties")
|> Enum.map(fn {name, config} ->
{Macro.underscore(name), config}
end)
|> Enum.sort_by(fn {name, _} ->
name not in List.wrap(config[:primary_key])
end)
|> Enum.map_join("\n\n", fn {name, property} ->
type =
case property do
%{"enum" => _values} ->
":atom"
%{"description" => _description} ->
":string"
%{"format" => "date-time"} ->
":utc_datetime"
%{"type" => "string"} ->
":string"
%{"type" => "object"} ->
":map"
%{"type" => "array"} ->
":map"
%{"type" => "integer"} ->
":integer"
%{"type" => "boolean"} ->
":boolean"
other ->
raise "Unsupported property: #{inspect(other)}"
end
constraints =
case property do
%{"enum" => values} ->
"one_of: #{inspect(Enum.map(values, &String.to_atom/1))}"
%{"maxLength" => max, "minLength" => min, "type" => "string"} ->
"min_length: #{min}, max_length: #{max}"
%{"maxLength" => max, "type" => "string"} ->
"max_length: #{max}"
%{"minLength" => min, "type" => "string"} ->
"min_length: #{min}"
_ ->
nil
end
primary_key? = name in List.wrap(config[:primary_key])
if constraints || primary_key? do
constraints =
if constraints do
"constraints #{constraints}"
end
primary_key =
if primary_key? do
"""
primary_key? true
allow_nil? false
"""
end
"""
attribute :#{name}, #{type} do
#{primary_key}
#{constraints}
end
"""
else
"""
attribute :#{name}, #{type}
"""
end
end)
tesla =
if main_config[:tesla] do
"tesla #{main_config[:tesla]}"
end
endpoint =
if main_config[:endpoint] do
"base \"#{main_config[:endpoint]}\""
end
code =
case config[:path] do
"__schema__" ->
"""
defmodule #{resource} do
use Ash.Resource, domain: #{inspect(domain)}, data_layer: AshJsonApiWrapper.DataLayer
actions do
#{actions}
end
attributes do
#{attributes}
end
end
"""
|> Code.format_string!()
|> IO.iodata_to_binary()
_ ->
"""
defmodule #{resource} do
use Ash.Resource, domain: #{inspect(domain)}, data_layer: AshJsonApiWrapper.DataLayer
json_api_wrapper do
#{tesla}
if #{endpoints} = ""
endpoints do
#{endpoint}
#{endpoints}
end
#{fields}
end
actions do
#{actions}
end
attributes do
#{attributes}
end
end
"""
|> Code.format_string!()
|> IO.iodata_to_binary()
end
{resource, code}
end)
end
defp operation_id(%{"operationId" => operationId}) do
operationId
|> Macro.underscore()
end
defp operations(json, config) do
json["paths"]
|> Enum.filter(fn {path, _value} ->
String.starts_with?(path, config[:path])
end)
|> Enum.flat_map(fn {path, methods} ->
Enum.map(methods, fn {method, config} ->
{path, method, config}
end)
end)
|> Enum.filter(fn {_path, method, _config} ->
method in ["get", "post"]
end)
end
end
Generated files
Running the mix generates the resources and the files.
"ecommerce_api_address"
"ecommerce_api_department"
"ecommerce_api_ecommerce_address"
"ecommerce_api_ecommerce_customer"
"ecommerce_api_ecommerce_discount"
"ecommerce_api_ecommerce_order_line_item"
"ecommerce_api_tracking_item"
"ecommerce_api_website"
"ecommerce_api_ecommerce_store"
"ecommerce_api_ecommerce_order"
"ecommerce_api_ecommerce_customer"
"ecommerce_api_ecommerce_product"Generated Resource
Currently the JSON Wrappers design is to create a simple Ash Phoenix resource which uses JSON to read and write data back to the Ecommerce API. I am looking to extend this to do more generation, which is coming soon.
defmodule EcommerceApiEcommerceProduct do
use Ash.Resource, domain: API.Gen.Ecommerce.Domain, data_layer: AshJsonApiWrapper.DataLayer
json_api_wrapper do
tesla(Elixir.TestingTesla)
endpoints do
base("https://developers.apideck.com")
endpoint :products_one do
path("/ecommerce/products/{id}")
end
end
fields do
field :orderId do
filter_handler({:place_in_csv_list, ["id"]})
end
end
end
actions do
read(:products_one)
end
attributes do
attribute :id, :string do
primary_key?(true)
allow_nil?(false)
end
attribute(:categories, :string)
attribute(:created_at, :string)
attribute(:description, :string)
attribute(:images, :string)
attribute(:inventory_quantity, :string)
attribute(:name, :string)
attribute(:options, :string)
attribute(:price, :string)
attribute(:sku, :string)
attribute :status, :atom do
constraints(one_of: [:active, :archived])
end
attribute(:tags, :string)
attribute(:updated_at, :string)
attribute(:variants, :map)
attribute(:weight, :string)
attribute(:weight_unit, :string)
end
end
