Use AWS Serverless to sell items in Habitica

30 June 2024

I use Habitica daily. It is a very strong motivator to keep me on track with my daily tasks. However, I already collected all the free pets and mounts and so I don't need most of the items I collect. Selling them is a pain as you have to sell one item with each click. But Habitica has a remarkable API. I decided to automate this process with AWS Lambda and EventBridge scheduled events.

Header

Getting API key

Obviously you need a Habitica account. Once you are there (on a desktop browser), go to Settings -> Site Data. You will see API section where you can download the token. But be careful as the only way to reset the token is to send email to Habitica support. Also save your user ID.

API Token

Storing token in AWS Secrets Manager

Secrets Manager is a great place to store such a token. Alternatively you can use SSM Parameter Store with SecretString to keep costs low. In a new directory for our project we will create template.yaml. This will be our AWS SAM template. Normally SAM doesn't have a resource for Secrets Manager. However, it is just a CloudFormation file with extras so we can use all CloudFormation compatible resources.

The user ID and API key will be provided as secret parameters (NoEcho). You will need to specify them every time you use sam deploy --guided.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: A function that sells Habitica items periodically leaving only a subset of those

Parameters:
  HabiticaApiKey:
    Type: String
    Description: The API key for the Habitica account
    NoEcho: true
  HabiticaUserId:
    Type: String
    Description: The user ID for the Habitica account
    NoEcho: true

Resources:
  HabiticaSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: Secret for the Habitica API key and user ID
      SecretString: !Sub '{"${HabiticaUserId}": "${HabiticaApiKey}"}'

Authenticating with Habitica

So using the provided values we can authenticate with Habitica API by providing them as headers. We will define our functions with Python. I created the file below in a new directory under sell_items/auth.py. The first function is used to retrieve the user and key from Secrets Manager by specified ARN in the environment variables. If this is not specified, it tries resolving the secret and user directly from the environment variables (this can be used for testing locally).

The secret is defined in Secrets Manager as a JSON object where the key is the user ID and the value is the API token. get_secret function will return the first key-value pair as a tuple. The x-client header is based on user ID as per Habitica requirements when creating a third-party integration - see this wiki page.

import os, json
from boto3 import client

def get_secret(habitica):
    """
    Retrieves the secret from AWS Secrets Manager.
    """
    secrets = client("secretsmanager")
    secret = secrets.get_secret_value(SecretId=habitica)
    secret = json.loads(secret['SecretString'])
    return list(secret.items())[0]


def get_headers() -> dict:
    """
    Produces a list of headers that are required for Habitica API requests.
    """
    # By default try to load environment variables.
    HABITICA_USER=os.getenv('HABITICA_USER')
    HABITICA_KEY=os.getenv('HABITICA_KEY')

    # But if there's HABITICA_SECRET set, use Secrets Manager.
    HABITICA_SECRET=os.getenv('HABITICA_SECRET')
    if HABITICA_SECRET:
        print('Using Secrets Manager.')
        HABITICA_USER, HABITICA_KEY = get_secret(HABITICA_SECRET)

    HABITICA_CLIENT=f"{HABITICA_USER}-itemseller10"
    print(f"Client {HABITICA_CLIENT}")

    return {
        "x-api-user": HABITICA_USER,
        "x-api-key": HABITICA_KEY,
        "x-client": HABITICA_CLIENT
    }

Retrieving data from the Habitica API

In order to test the authentication, we can write a function that will get the inventory that we will later use in the application. This function will just retrieve all hatching potions, eggs and food from inventory along with gold. In case the authentication fails, we will throw an exception.

import requests

HABITICA_URL="https://habitica.com/api/v3"

def get_inventory(headers: dict) -> tuple[dict, float]:
    """
    Gets user inventory in the form of eggs, potions, food and stats (gold).
    Returns a tuple of (items: dict, gold: float).
    """
    url = f"{HABITICA_URL}/user?userFields=stats,items.eggs,items.hatchingPotions,items.food"
    response = requests.get(url, headers=headers)
    code = response.status_code
    if code == 200:
        items = response.json()['data']['items']
        gold = response.json()['data']['stats']['gp']
        return items, gold
    raise Exception(response.json()['message'])

Next in a new file file main.py let us glue the two function together and print out the result. Set the values for HABITICA_SECRET and HABITICA_USER as environment variables.

from auth import get_headers
from actions import get_inventory

if __name__ == "__main__":
    headers = get_headers()
    items, gold = get_inventory(headers)
    print(f"Items: {items}")
    print(f"Gold: {gold}")

To run it safely without exposing your API token too much, use the following bash commands. It will ask you for the token without any prompt and won't echo.

$ read -s HABITICA_USER
$ read -s HABITICA_KEY
$ export HABITICA_USER HABITICA_KEY
$ python3 main.py
Client 12345678-8b3d-48af-aabb-ccddee445566-itemseller10
Gold: 22670.394004083322
Items: {'eggs': {'BearCub': 100, 'TigerCub': 100, 'PandaCub': 100, 'Fox': 100,...

Another important thing in Habitica is that some of the items are not obtainable randomly but are very valuable, purchased with gems. These items we want to protect from our script. I created a function that will filter items to leave only the most common ones. It is a simple Python array operation that you can find here.

Integrate the function in your get_inventory function.

from filters import filter_items
# ...
    if code == 200:
        items = response.json()['data']['items']
        gold = response.json()['data']['stats']['gp']
        return filter_items(items), gold
# ...

Functions for selling an item

Now we can write a function that will sell an item. This will be similarly just a REST call to Habitica. Compared to the UI available on the web or mobile, this function allows us to set the amount of items to sell.

def sell_item(headers: dict, item_type: str, item_key: str, count: int):
    """
    Sells a specific item with the given count.
    1:1 mapping of Habitica API documentation.
    """
    url = f"{HABITICA_URL}/user/sell/{item_type}/{item_key}?amount={count}"
    response = requests.post(url, headers=headers)
    code = response.status_code
    if code == 200:
        return response.json()
    raise Exception(response.json()['message'])

Find the most common item in your inventory and sell one of it for testing. Add the sell routine to the main section. This function will return a lot of data about the player including inventory after sale and current gold amount.

from auth import get_headers
from actions import get_inventory, sell_item

if __name__ == "__main__":
    headers = get_headers()
    items, gold = get_inventory(headers)
    print(f"Gold: {gold:.2f}\nItems: {items}")

    sale = sell_item(headers, 'eggs', 'TigerCub', 1)
    new_gold = sale['data']['stats']['gp']
    print(f"Gold after sale: {new_gold:.2f}")

Run the script again and see how much money you have after a sale.

$ python3 main.py
Client 12345678-8b3d-48af-aabb-ccddee445566-itemseller10
Gold: 22824.60
Items: {'eggs': {'BearCub': 101, 'TigerCub': 99, 'PandaCub': 100, ...
Gold after sale: 22827.60

Iterating through the items

Now, we can iterate through all the items and sell them. However, we have a lot of item types and we don't want to hit Habitica's API rate threshold, which is very low. In order to avoid this, we will sell only items from one category in one run of the function. The category will be passed to Lambda in the event. The new function will take event as an argument.

Next thing is that we don't want to sell all the items. This fact can help us improve the function even more by filtering out items that don't need to be sold as we have less of them then the value of items to keep. The function will return how much gold we made for this function invocation.

def sell_items(headers: dict, items_to_leave: int, event) -> float:
    """
    Sells items of a specific category in `event['sell_category']`.
    """
    if not is_event_valid(event):
        print("No sell_category specified!")
        raise ValueError("Missing 'sell_category' key in event. And such, ignoring this event.")

    category = event['sell_category']

    items, gold = get_inventory(headers)
    new_gold = gold     # To define it in case we don't sell anything.
    print(f"Starting with gold [{category}] {gold:.2f}.")
    items_to_sell = items[category]

    # Filter items that we have more than threshold.
    items_to_sell = {k: v for k, v in items_to_sell.items() if v > items_to_leave}

    # Print history of what is being done.
    string_items = ", ".join([f"{k} ({v})" for k, v in items_to_sell.items()])
    print(f"Items to sell [{category}]: {string_items}")

    last_sale = None
    # Sell items and store the last sale for gold calculation.
    for item, amount in items_to_sell.items():
        count = amount - items_to_leave
        print(f"Selling {count} of {item}.")
        try:
            _sold = sell_item(headers, event['sell_category'], item, count)
        except Exception as e:
            print(f"Failed to sell {item}: {e}.")
            continue
        last_sale = _sold
        print(f"Sold {count} of {item}.")

    if last_sale is not None:
        new_gold = last_sale['data']['stats']['gp']
        print(f"Gold after sale [{category}]: {new_gold:.2f}. Made {new_gold - gold:.2f} gold.")
    else:
        print(f"Didn't sell anything for {category}.")

    return new_gold - gold

Main function for Lambda

Now we need to glue together all the parts. In main.py we can import required functions that we created. We will also initialize all the constants such as API key, user ID, environment configuration. Remember to remove everything under if __name__ == "__main__": as it is not needed in Lambda and can be called accidentally.

from actions import sell_items
from auth import get_headers
import os, json

HEADERS = get_headers()
# This is the amount of each item that will be left and not sold.
ITEMS_TO_LEAVE = int(os.getenv('ITEMS_TO_LEAVE', "150"))

def lambda_handler(event, context):
    if event is str:
        event: dict = json.loads(event)

    try:
        gp = sell_items(HEADERS, ITEMS_TO_LEAVE, event)
    except Exception as e:
        return {
            "statusCode": 400,
            "body": str(e)
        }

    return {
        "statusCode": 200,
        "body": "Earned {gp:.2f} gold."
    }

Adding function to SAM template

SAM is perfect for handling Lambda functions. It can install requirements.txt for us, zip everything and deploy. In the scripts I wrote above, only requests is needed as an extra package. If you prefer to keep your function slim, you can use urllib3 alternatively. Otherwise, create requirements.txt in your code directory (my is sell_items) with just one line: requests.

We will also specify Lambda memory and execution timeout. Add lines below to your template.yaml under the description.

# ...
Globals:
  Function:
    Timeout: 15
    MemorySize: 128

In Resources section we will add the Lambda. It will reference the Secret so it's worth putting it below the HabiticaSecret block. In HABITICA_SECRET environment variable we will have ARN of the secret that will be picked up by our function. In Policies we will also specify IAM permissions for Lambda's execution role so that it can read the secret. AWSSecretsManagerGetSecretValuePolicy is a managed policy that will put all the necessary permissions, without us needing to refer the documentation 😁. I recommend keeping architecture as arm64 as it is more cost efficient. CodeUri refers to the directory where you keep all your Python files.

Resources:
  # ...
  HabiticaSellItems:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: sell_items/
      Handler: main.lambda_handler
      Runtime: python3.12
      Architectures:
        - arm64
      Policies:
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref HabiticaSecret
      Environment:
        Variables:
          HABITICA_SECRET: !Ref HabiticaSecret
          ITEMS_TO_LEAVE: "100"

Scheduling the function

As you remember from previous sections, Habitica's API has a strict rate limit. To mitigate this, we split the selling routines into categories. For each category we will define a separate EventBridge schedule that will fire at a different time. Append Events section to you Function resource. For each event definition we will specify a different input to be feed to the function.

 HabiticaSellItems:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: sell_items/
      #...
      Events:
        FoodSchedule:
          Type: Schedule
          Properties:
            Schedule: cron(10 1 * * ? *)
            Input: '{"sell_category": "food"}'
            Enabled: true
        EggsSchedule:
          Type: Schedule
          Properties:
            Schedule: cron(15 1 * * ? *)
            Input: '{"sell_category": "eggs"}'
            Enabled: true
        HatchingPotionsSchedule:
          Type: Schedule
          Properties:
            Schedule: cron(20 1 * * ? *)
            Input: '{"sell_category": "hatchingPotions"}'
            Enabled: true

The above configuration will run the tasks at 1:10AM UTC, 1:15 and 1:20. Next we will build the configuration using SAM and deploy it as well. You will need your AWS access keys and SAM installed. For the first run use sam deploy --guided to give Habitica API key and user. If you don't change the user or API key, just use sam deploy and it will skip asking you for parameters of the secret.

$ sam build
$ sam deploy --guided

The ready project is available on GitHub tagged v1.

Watching the results

So I didn't specify ITEMS_TO_LEAVE at the beginning so it defaulted to 150. It successfully ran at night and sold all the items I had more than 150 of. For the next night I lowered the threshold to 100 and again it worked perfectly.

Sold items every day

I also checked CloudWatch to see the logs of the function and the whole process. Here everything seemed stable and behaved as expected. First selling down to 150 things in the inventory and the next day to 100.

CloudWatch logs day 1

CloudWatch logs day 2

pabis.eu