fastapi集成elastic-apm，实现性能监控

本文适用于 Starlette/FastAPI

软件版本背景：
elastic 8.9
apm 6.x

官方原文如下：

Starlette/FastAPI Support

Incorporating Elastic APM into your Starlette project only requires a few easy steps.

Installation

Install the Elastic APM agent using pip:

$ pip install elastic-apm

or add elastic-apm to your project’s requirements.txt file.

Setup

To set up the agent, you need to initialize it with appropriate settings.

The settings are configured either via environment variables, or as initialization arguments.

You can find a list of all available settings in the Configuration page.

To initialize the agent for your application using environment variables, add the ElasticAPM middleware to your Starlette application:

扫描二维码关注公众号，回复： 17062061 查看本文章

from starlette.applications import Starlette
from elasticapm.contrib.starlette import ElasticAPM

app = Starlette()
app.add_middleware(ElasticAPM)

Warning	If you are using any `BaseHTTPMiddleware` middleware, you must add them before the ElasticAPM middleware. This is because `BaseHTTPMiddleware` breaks `contextvar` propagation, as noted here.

To configure the agent using initialization arguments:

from starlette.applications import Starlette
from elasticapm.contrib.starlette import make_apm_client, ElasticAPM

apm = make_apm_client({
    'SERVICE_NAME': '<SERVICE-NAME>',
    'SECRET_TOKEN': '<SECRET-TOKEN>',
})
app = Starlette()
app.add_middleware(ElasticAPM, client=apm)

FastAPI

Because FastAPI supports Starlette middleware, using the agent with FastAPI is almost exactly the same as with Starlette:

from fastapi import FastAPI
from elasticapm.contrib.starlette ElasticAPM

app = FastAPI()
app.add_middleware(ElasticAPM)

Usage

Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within starlette.

Capture an arbitrary exception by calling capture_exception:

try:
    1 / 0
except ZeroDivisionError:
    apm.client.capture_exception()

Log a generic message with capture_message:

apm.client.capture_message('hello, world!')

Performance metrics

If you’ve followed the instructions above, the agent has installed our instrumentation middleware which will process all requests through your app. This will measure response times, as well as detailed performance data for all supported technologies.

Note	Due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited.

Ignoring specific routes

You can use the TRANSACTIONS_IGNORE_PATTERNS configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name:

apm = make_apm_client({
    # ...
    'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret']
    # ...
})

This would ignore any requests using the GET /secret route and any requests containing /extra_secret.

Supported Starlette and Python versions

A list of supported Starlette and Python versions can be found on our Supported Technologies page.

Note	Elastic APM only supports `asyncio` when using Python 3.7+

具体使用示例

在这里插入图片描述

在加入了以上的代码配置后，按照正常启动应用的步骤就可以在kibana的apm界面看到对应的服务监控了。

关于配置文件，可以参考 elastic-apm包下的文件
venv/lib/python3.7/site-packages/elasticapm/conf/init.py 中的 class Config，其中全部大写的这些字段就是配置项

class Config(_ConfigBase):
    service_name = _ConfigValue(
        "SERVICE_NAME",
        validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
        default="unknown-python-service",
        required=True,
    )
    service_node_name = _ConfigValue("SERVICE_NODE_NAME")
    environment = _ConfigValue("ENVIRONMENT")
    transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
    secret_token = _ConfigValue("SECRET_TOKEN")
    api_key = _ConfigValue("API_KEY")
    debug = _BoolConfigValue("DEBUG", default=False)
    server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
    server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
    server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
    verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
    use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
    include_paths = _ListConfigValue("INCLUDE_PATHS")
    exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
    filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
    server_timeout = _ConfigValue(
        "SERVER_TIMEOUT",
        type=float,
        validators=[
            UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {
    
    "ms": 0.001, "s": 1, "m": 60, None: 1000})
        ],
        default=5,
    )
    hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
    auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
    transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
    processors = _ListConfigValue(
        "PROCESSORS",
        default=[
            "elasticapm.processors.sanitize_stacktrace_locals",
            "elasticapm.processors.sanitize_http_request_cookies",
            "elasticapm.processors.sanitize_http_response_cookies",
            "elasticapm.processors.sanitize_http_headers",
            "elasticapm.processors.sanitize_http_wsgi_env",
            "elasticapm.processors.sanitize_http_request_body",
        ],
    )
    sanitize_field_names = _ListConfigValue(
        "SANITIZE_FIELD_NAMES", type=starmatch_to_regex, default=BASE_SANITIZE_FIELD_NAMES
    )
    metrics_sets = _ListConfigValue(
        "METRICS_SETS",
        default=[
            "elasticapm.metrics.sets.cpu.CPUMetricSet",
        ],
    )
    metrics_interval = _DurationConfigValue(
        "METRICS_INTERVAL",
        validators=[ExcludeRangeValidator(0.001, 0.999, "{range_start} - {range_end} s")],
        default=timedelta(seconds=30),
    )
    breakdown_metrics = _BoolConfigValue("BREAKDOWN_METRICS", default=True)
    prometheus_metrics = _class Config(_ConfigBase):
    service_name = _ConfigValue(
        "SERVICE_NAME",
        validators=[RegexValidator("^[a-zA-Z0-9 _-]+$")],
        default="unknown-python-service",
        required=True,
    )
    service_node_name = _ConfigValue("SERVICE_NODE_NAME")
    environment = _ConfigValue("ENVIRONMENT")
    transport_json_serializer = _ConfigValue("TRANSPORT_JSON_SERIALIZER")
    secret_token = _ConfigValue("SECRET_TOKEN")
    api_key = _ConfigValue("API_KEY")
    debug = _BoolConfigValue("DEBUG", default=False)
    server_url = _ConfigValue("SERVER_URL", default="http://127.0.0.1:8200", required=True)
    server_cert = _ConfigValue("SERVER_CERT", validators=[FileIsReadableValidator()])
    server_ca_cert_file = _ConfigValue("SERVER_CA_CERT_FILE", validators=[FileIsReadableValidator()])
    verify_server_cert = _BoolConfigValue("VERIFY_SERVER_CERT", default=True)
    use_certifi = _BoolConfigValue("USE_CERTIFI", default=True)
    include_paths = _ListConfigValue("INCLUDE_PATHS")
    exclude_paths = _ListConfigValue("EXCLUDE_PATHS", default=compat.get_default_library_patters())
    filter_exception_types = _ListConfigValue("FILTER_EXCEPTION_TYPES")
    server_timeout = _ConfigValue(
        "SERVER_TIMEOUT",
        type=float,
        validators=[
            UnitValidator(r"^((?:-)?\d+)(ms|s|m)?$", r"\d+(ms|s|m)", {
    
    "ms": 0.001, "s": 1, "m": 60, None: 1000})
        ],
        default=5,
    )
    hostname = _ConfigValue("HOSTNAME", default=socket.gethostname())
    auto_log_stacks = _BoolConfigValue("AUTO_LOG_STACKS", default=True)
    transport_class = _ConfigValue("TRANSPORT_CLASS", default="elasticapm.transport.http.Transport", required=True)
    processors = _ListConfigValue(
        "PROCESSORS",
        default=[
            "elasticapm.processors.sanitize_stacktrace_locals",
            "elasticapm.processors.sanitize_http_request_cookies",
            "elasticapm.processors.sanitize_http_response_cookies",
            "elasticapm.processors.sanitize_http_headers",
            "elasticapm.processooolConfigValue("PROMETHEUS_METRICS", default=False)
    prometheus_metrics_prefix = _ConfigValue("PROMETHEUS_METRICS_PREFIX", default="prometheus.metrics.")
    disable_metrics = _ListConfigValue("DISABLE_METRICS", type=starmatch_to_regex, default=[])
    central_config = _BoolConfigValue("CENTRAL_CONFIG", default=True)
    api_request_size = _ConfigValue("API_REQUEST_SIZE", type=int, validators=[size_validator], default=768 * 1024)
    api_request_time = _DurationConfigValue("API_REQUEST_TIME", default=timedelta(seconds=10))
    transaction_sample_rate = _ConfigValue(
        "TRANSACTION_SAMPLE_RATE", type=float, validators=[PrecisionValidator(4, 0.0001)], default=1.0
    )
    transaction_max_spans = _ConfigValue("TRANSACTION_MAX_SPANS", type=int, default=500)
    stack_trace_limit = _ConfigValue("STACK_TRACE_LIMIT", type=int, default=50)
    span_frames_min_duration = _DurationConfigValue(
        "SPAN_FRAMES_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
    )
    span_stack_trace_min_duration = _DurationConfigValue(
        "SPAN_STACK_TRACE_MIN_DURATION", default=timedelta(seconds=0.005), unitless_factor=0.001
    )
    span_compression_enabled = _BoolConfigValue("SPAN_COMPRESSION_ENABLED", default=True)
    span_compression_exact_match_max_duration = _DurationConfigValue(
        "SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION",
        default=timedelta(seconds=0.05),
    )
    span_compression_same_kind_max_duration = _DurationConfigValue(
        "SPAN_COMPRESSION_SAME_KIND_MAX_DURATION",
        default=timedelta(seconds=0),
    )
    exit_span_min_duration = _DurationConfigValue(
        "EXIT_SPAN_MIN_DURATION",
        default=timedelta(seconds=0),
    )
    collect_local_variables = _ConfigValue("COLLECT_LOCAL_VARIABLES", default="errors")
    source_lines_error_app_frames = _ConfigValue("SOURCE_LINES_ERROR_APP_FRAMES", type=int, default=5)
    source_lines_error_library_frames = _ConfigValue("SOURCE_LINES_ERROR_LIBRARY_FRAMES", type=int, default=5)
    source_lines_span_app_frames = _ConfigValue("SOURCE_LINES_SPAN_APP_FRAMES", type=int, default=0)
    source_lines_span_library_frames = _ConfigValue("SOURCE_LINES_SPAN_LIBRARY_FRAMES", type=int, default=0)
    local_var_max_length = _ConfigValue("LOCAL_VAR_MAX_LENGTH", type=int, default=200)
    local_var_list_max_length = _ConfigValue("LOCAL_VAR_LIST_MAX_LENGTH", type=int, default=10)
    local_var_dict_max_length = _ConfigValue("LOCAL_VAR_DICT_MAX_LENGTH", type=int, default=10)
    capture_body = _ConfigValue(
        "CAPTURE_BODY",
        default="off",
        validators=[lambda val, _: {"errors": "error", "transactions": "transaction"}.get(val, val)],
    )
    async_mode = _BoolConfigValue("ASYNC_MODE", default=True)
    instrument_django_middleware = _BoolConfigValue("INSTRUMENT_DJANGO_MIDDLEWARE", default=True)
    autoinsert_django_middleware = _BoolConfigValue("AUTOINSERT_DJANGO_MIDDLEWARE", default=True)
    transactions_ignore_patterns = _ListConfigValue("TRANSACTIONS_IGNORE_PATTERNS", default=[])
    transaction_ignore_urls = _ListConfigValue("TRANSACTION_IGNORE_URLS", type=starmatch_to_regex, default=[])
    ignore_message_queues = _ListConfigValue("IGNORE_MESSAGE_QUEUES", type=starmatch_to_regex, default=[])
    service_version = _ConfigValue("SERVICE_VERSION")
    framework_name = _ConfigValue("FRAMEWORK_NAME")
    framework_version = _ConfigValue("FRAMEWORK_VERSION")
    global_labels = _DictConfigValue("GLOBAL_LABELS")
    disable_send = _BoolConfigValue("DISABLE_SEND", default=False)
    enabled = _BoolConfigValue("ENABLED", default=True)
    recording = _BoolConfigValue("RECORDING", default=True)
    instrument = _BoolConfigValue("INSTRUMENT", default=True)
    enable_distributed_tracing = _BoolConfigValue("ENABLE_DISTRIBUTED_TRACING", default=True)
    capture_headers = _BoolConfigValue("CAPTURE_HEADERS", default=True)
    django_transaction_name_from_route = _BoolConfigValue("DJANGO_TRANSACTION_NAME_FROM_ROUTE", default=False)
    disable_log_record_factory = _BoolConfigValue("DISABLE_LOG_RECORD_FACTORY", default=False)
    use_elastic_traceparent_header = _BoolConfigValue("USE_ELASTIC_TRACEPARENT_HEADER", default=True)
    use_elastic_excepthook = _BoolConfigValue("USE_ELASTIC_EXCEPTHOOK", default=False)
    cloud_provider = _ConfigValue("CLOUD_PROVIDER", default=True)
    log_level = _ConfigValue(
        "LOG_LEVEL",
        validators=[EnumerationValidator(["trace", "debug", "info", "warning", "warn", "error", "critical", "off"])],
        callbacks=[_log_level_callback],
    )
    log_file = _ConfigValue("LOG_FILE", default="")
    log_file_size = _ConfigValue("LOG_FILE_SIZE", validators=[size_validator], type=int, default=50 * 1024 * 1024)
    log_ecs_reformatting = _ConfigValue(
        "LOG_ECS_REFORMATTING",
        validators=[EnumerationValidator(["off", "override"])],
        callbacks=[_log_ecs_reformatting_callback],
        default="off",
    )
    trace_continuation_strategy = _ConfigValue(
        "TRACE_CONTINUATION_STRATEGY",
        validators=[
            EnumerationValidator(
                [
                    TRACE_CONTINUATION_STRATEGY.CONTINUE,
                    TRACE_CONTINUATION_STRATEGY.RESTART,
                    TRACE_CONTINUATION_STRATEGY.RESTART_EXTERNAL,
                ]
            )
        ],
        default=TRACE_CONTINUATION_STRATEGY.CONTINUE,
    )
    include_process_args = _BoolConfigValue("INCLUDE_PROCESS_ARGS", default=False)

官网的配置项介绍

https://www.elastic.co/guide/en/apm/agent/python/6.x/configuration.html

各语言框架集成elastic-apm官网介绍

https://www.elastic.co/guide/en/apm/guide/current/_step_3_install_apm_agents.html