fastapi integrates elastic-apm to implement performance monitoring

This article applies to Starlette/FastAPI

Software version background:
elastic 8.9
apm 6.x

The official text is as follows:

Starlette/FastAPI Support

Incorporating Elastic APM into your Starlette project only requires a few easy steps.

Installation

Install the Elastic APM agent using pip:

$ pip install elastic-apm

or add elastic-apm to your project’s requirements.txt file.

Setup

To set up the agent, you need to initialize it with appropriate settings.

The settings are configured either via environment variables, or as initialization arguments.

You can find a list of all available settings in the Configuration page.

To initialize the agent for your application using environment variables, add the ElasticAPM middleware to your Starlette application:

from starlette.applications import Starlette
from elasticapm.contrib.starlette import ElasticAPM

app = Starlette()
app.add_middleware(ElasticAPM)

Warning	If you are using any `BaseHTTPMiddleware` middleware, you must add them before the ElasticAPM middleware. This is because `BaseHTTPMiddleware` breaks `contextvar` propagation, as noted here.

To configure the agent using initialization arguments:

from starlette.applications import Starlette
from elasticapm.contrib.starlette import make_apm_client, ElasticAPM

apm = make_apm_client({
    'SERVICE_NAME': '<SERVICE-NAME>',
    'SECRET_TOKEN': '<SECRET-TOKEN>',
})
app = Starlette()
app.add_middleware(ElasticAPM, client=apm)

FastAPI

Because FastAPI supports Starlette middleware, using the agent with FastAPI is almost exactly the same as with Starlette:

from fastapi import FastAPI
from elasticapm.contrib.starlette ElasticAPM

app = FastAPI()
app.add_middleware(ElasticAPM)

Usage

Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within starlette.

Capture an arbitrary exception by calling capture_exception:

try:
    1 / 0
except ZeroDivisionError:
    apm.client.capture_exception()

Log a generic message with capture_message:

apm.client.capture_message('hello, world!')

Performance metrics

If you’ve followed the instructions above, the agent has installed our instrumentation middleware which will process all requests through your app. This will measure response times, as well as detailed performance data for all supported technologies.

Note	Due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited.

Ignoring specific routes

You can use the TRANSACTIONS_IGNORE_PATTERNS configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name:

apm = make_apm_client({
    # ...
    'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret']
    # ...
})

This would ignore any requests using the GET /secret route and any requests containing /extra_secret.

Supported Starlette and Python versions

A list of supported Starlette and Python versions can be found on our Supported Technologies page.

Note	Elastic APM only supports `asyncio` when using Python 3.7+

Specific usage examples

Insert image description here

After adding the above code configuration, you can see the corresponding service monitoring in kibana's apm interface by following the steps of starting the application normally.

Regarding the configuration file, you can refer to the class Config in the file venv/lib/python3.7/site-packages/elasticapm/conf/ init.py under the elastic-apm package . The fields in all capital letters are the configuration items.

class Config(_ConfigBase):
    service_name = _ConfigValue(
        "SERVICE_NAME",
        validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
        default="unknown-python-service",
        required=True,
    )
    service_node_name = _ConfigValue("SERVICE_NODE_NAME")
    environment = _ConfigValue("ENVIRONMENT")
    transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
    secret_token = _ConfigValue("SECRET_TOKEN")
    api_key = _ConfigValue("API_KEY")
    debug = _BoolConfigValue("DEBUG", default=False)
    server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
    server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
    server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
    verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
    use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
    include_paths = _ListConfigValue("INCLUDE_PATHS")
    exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
    filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
    server_timeout = _ConfigValue(
        "SERVER_TIMEOUT",
        type=float,
        validators=[
            UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {
    
    "ms": 0.001, "s": 1, "m": 60, None: 1000})
        ],
        default=5,
    )
    hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
    auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
    transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
    processors = _ListConfigValue(
        "PROCESSORS",
        default=[
            "elasticapm.processors.sanitize_stacktrace_locals",
            "elasticapm.processors.sanitize_http_request_cookies",
            "elasticapm.processors.sanitize_http_response_cookies",
            "elasticapm.processors.sanitize_http_headers",
            "elasticapm.processors.sanitize_http_wsgi_env",
            "elasticapm.processors.sanitize_http_request_body",
        ],
    )
    sanitize_field_names = _ListConfigValue(
        "SANITIZE_FIELD_NAMES", type=starmatch_to_regex, default=BASE_SANITIZE_FIELD_NAMES
    )
    metrics_sets = _ListConfigValue(
        "METRICS_SETS",
        default=[
            "elasticapm.metrics.sets.cpu.CPUMetricSet",
        ],
    )
    metrics_interval = _DurationConfigValue(
        "METRICS_INTERVAL",
        validators=[ExcludeRangeValidator(0.001, 0.999, "{range_start} - {range_end} s")],
        default=timedelta(seconds=30),
    )
    breakdown_metrics = _BoolConfigValue("BREAKDOWN_METRICS", default=True)
    prometheus_metrics = _class Config(_ConfigBase):
    service_name = _ConfigValue(
        "SERVICE_NAME",
        validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
        default="unknown-python-service",
        required=True,
    )
    service_node_name = _ConfigValue("SERVICE_NODE_NAME")
    environment = _ConfigValue("ENVIRONMENT")
    transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
    secret_token = _ConfigValue("SECRET_TOKEN")
    api_key = _ConfigValue("API_KEY")
    debug = _BoolConfigValue("DEBUG", default=False)
    server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
    server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
    server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
    verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
    use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
    include_paths = _ListConfigValue("INCLUDE_PATHS")
    exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
    filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
    server_timeout = _ConfigValue(
        "SERVER_TIMEOUT",
        type=float,
        validators=[
            UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {
    
    "ms": 0.001, "s": 1, "m": 60, None: 1000})
        ],
        default=5,
    )
    hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
    auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
    transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
    processors = _ListConfigValue(
        "PROCESSORS",
        default=[
            "elasticapm.processors.sanitize_stacktrace_locals",
            "elasticapm.processors.sanitize_http_request_cookies",
            "elasticapm.processors.sanitize_http_response_cookies",
            "elasticapm.processors.sanitize_http_headers",
            "elasticapm.processooolConfigValue("PROMETHEUS_METRICS", default=False)
    prometheus_metrics_prefix = _ConfigValue("PROMETHEUS_METRICS_PREFIX", default="prometheus.metrics.")
    disable_metrics = _ListConfigValue("DISABLE_METRICS", type=starmatch_to_regex, default=[])
    central_config = _BoolConfigValue("CENTRAL_CONFIG", default=True)
    api_request_size = _ConfigValue("API_REQUEST_SIZE", type=int, validators=[size_validator], default=768 * 1024)
    api_request_time = _DurationConfigValue("API_REQUEST_TIME", default=timedelta(seconds=10))
    transaction_sample_rate = _ConfigValue(
        "TRANSACTION_SAMPLE_RATE", type=float, validators=[PrecisionValidator(4, 0.0001)], default=1.0
    )
    transaction_max_spans = _ConfigValue("TRANSACTION_MAX_SPANS", type=int, default=500)
    stack_trace_limit = _ConfigValue("STACK_TRACE_LIMIT", type=int, default=50)
    span_frames_min_duration = _DurationConfigValue(
        "SPAN_FRAMES_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
    )
    span_stack_trace_min_duration = _DurationConfigValue(
        "SPAN_STACK_TRACE_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
    )
    span_compression_enabled = _BoolConfigValue("SPAN_COMPRESSION_ENABLED", default=True)
    span_compression_exact_match_max_duration = _DurationConfigValue(
        "SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION",
        default=timedelta(seconds=0.05),
    )
    span_compression_same_kind_max_duration = _DurationConfigValue(
        "SPAN_COMPRESSION_SAME_KIND_MAX_DURATION",
        default=timedelta(seconds=0),
    )
    exit_span_min_duration = _DurationConfigValue(
        "EXIT_SPAN_MIN_DURATION",
        default=timedelta(seconds=0),
    )
    collect_local_variables = _ConfigValue("COLLECT_LOCAL_VARIABLES", default="errors")
    source_lines_error_app_frames = _ConfigValue("SOURCE_LINES_ERROR_APP_FRAMES", type=int, default=5)
    source_lines_error_library_frames = _ConfigValue("SOURCE_LINES_ERROR_LIBRARY_FRAMES", type=int, default=5)
    source_lines_span_app_frames = _ConfigValue("SOURCE_LINES_SPAN_APP_FRAMES", type=int, default=0)
    source_lines_span_library_frames = _ConfigValue("SOURCE_LINES_SPAN_LIBRARY_FRAMES", type=int, default=0)
    local_var_max_length = _ConfigValue("LOCAL_VAR_MAX_LENGTH", type=int, default=200)
    local_var_list_max_length = _ConfigValue("LOCAL_VAR_LIST_MAX_LENGTH", type=int, default=10)
    local_var_dict_max_length = _ConfigValue("LOCAL_VAR_DICT_MAX_LENGTH", type=int, default=10)
    capture_body = _ConfigValue(
        "CAPTURE_BODY",
        default="off",
        validators=[lambda val, _: {"errors": "error", "transactions": "transaction"}.get(val, val)],
    )
    async_mode = _BoolConfigValue("ASYNC_MODE", default=True)
    instrument_django_middleware = _BoolConfigValue("INSTRUMENT_DJANGO_MIDDLEWARE", default=True)
    autoinsert_django_middleware = _BoolConfigValue("AUTOINSERT_DJANGO_MIDDLEWARE", default=True)
    transactions_ignore_patterns = _ListConfigValue("TRANSACTIONS_IGNORE_PATTERNS", default=[])
    transaction_ignore_urls = _ListConfigValue("TRANSACTION_IGNORE_URLS", type=starmatch_to_regex, default=[])
    ignore_message_queues = _ListConfigValue("IGNORE_MESSAGE_QUEUES", type=starmatch_to_regex, default=[])
    service_version = _ConfigValue("SERVICE_VERSION")
    framework_name = _ConfigValue("FRAMEWORK_NAME")
    framework_version = _ConfigValue("FRAMEWORK_VERSION")
    global_labels = _DictConfigValue("GLOBAL_LABELS")
    disable_send = _BoolConfigValue("DISABLE_SEND", default=False)
    enabled = _BoolConfigValue("ENABLED", default=True)
    recording = _BoolConfigValue("RECORDING", default=True)
    instrument = _BoolConfigValue("INSTRUMENT", default=True)
    enable_distributed_tracing = _BoolConfigValue("ENABLE_DISTRIBUTED_TRACING", default=True)
    capture_headers = _BoolConfigValue("CAPTURE_HEADERS", default=True)
    django_transaction_name_from_route = _BoolConfigValue("DJANGO_TRANSACTION_NAME_FROM_ROUTE", default=False)
    disable_log_record_factory = _BoolConfigValue("DISABLE_LOG_RECORD_FACTORY", default=False)
    use_elastic_traceparent_header = _BoolConfigValue("USE_ELASTIC_TRACEPARENT_HEADER", default=True)
    use_elastic_excepthook = _BoolConfigValue("USE_ELASTIC_EXCEPTHOOK", default=False)
    cloud_provider = _ConfigValue("CLOUD_PROVIDER", default=True)
    log_level = _ConfigValue(
        "LOG_LEVEL",
        validators=[EnumerationValidator(["trace", "debug", "info", "warning", "warn", "error", "critical", "off"])],
        callbacks=[_log_level_callback],
    )
    log_file = _ConfigValue("LOG_FILE", default="")
    log_file_size = _ConfigValue("LOG_FILE_SIZE", validators=[size_validator], type=int, default=50 * 1024 * 1024)
    log_ecs_reformatting = _ConfigValue(
        "LOG_ECS_REFORMATTING",
        validators=[EnumerationValidator(["off", "override"])],
        callbacks=[_log_ecs_reformatting_callback],
        default="off",
    )
    trace_continuation_strategy = _ConfigValue(
        "TRACE_CONTINUATION_STRATEGY",
        validators=[
            EnumerationValidator(
                [
                    TRACE_CONTINUATION_STRATEGY.CONTINUE,
                    TRACE_CONTINUATION_STRATEGY.RESTART,
                    TRACE_CONTINUATION_STRATEGY.RESTART_EXTERNAL,
                ]
            )
        ],
        default=TRACE_CONTINUATION_STRATEGY.CONTINUE,
    )
    include_process_args = _BoolConfigValue("INCLUDE_PROCESS_ARGS", default=False)

Introduction to configuration items on the official website

https://www.elastic.co/guide/en/apm/agent/python/6.x/configuration.html

Introduction to the official website of elastic-apm integrating each language framework

https://www.elastic.co/guide/en/apm/guide/current/_step_3_install_apm_agents.html