crm-backend-matt/.qoder/repowiki/zh/content/故障排查.md

# 故障排查

<cite>
**本文引用的文件**   
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
- [application.yml](file://crm-app/src/main/resources/application.yml)
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [SecurityConfig.java](file://crm-auth/src/main/java/com/crm/auth/config/SecurityConfig.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [SecurityExceptionHandlers.java](file://crm-auth/src/main/java/com/crm/auth/security/SecurityExceptionHandlers.java)
- [AuthUserServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/AuthUserServiceImpl.java)
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [DataScopeInterceptor.java](file://crm-auth/src/main/java/com/crm/auth/security/DataScopeInterceptor.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
- [PermissionErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/PermissionErrorException.java)
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
- [TraceIdFilter.java](file://crm-base/src/main/java/com/crm/base/filter/TraceIdFilter.java)
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
- [application.yml](file://crm-auth/src/main/resources/application.yml)
</cite>

## 目录
1. [简介](#简介)
2. [项目结构](#项目结构)
3. [核心组件](#核心组件)
4. [架构总览](#架构总览)
5. [详细组件分析](#详细组件分析)
6. [依赖关系分析](#依赖关系分析)
7. [性能与稳定性](#性能与稳定性)
8. [故障排查指南](#故障排查指南)
9. [结论](#结论)
10. [附录：常用命令与工具](#附录常用命令与工具)

## 简介
本故障排查文档面向运维与开发人员，聚焦认证失败、权限拒绝、文件上传失败、数据库连接异常等典型问题，提供系统化的诊断方法与修复步骤。同时覆盖日志分析技巧、性能监控指标、内存泄漏检测、线程死锁排查等高级调试技能，并汇总常用命令、监控工具与日志查询语句，帮助快速定位和解决问题，降低停机时间。

## 项目结构
本项目采用多模块结构：
- crm-app：应用启动入口与全局配置
- crm-auth：认证授权与安全相关能力（登录、鉴权、数据权限）
- crm-base：公共基础能力（统一异常、结果封装、拦截器、通用配置）
- crm-file：文件服务（上传、下载、分片、预览、清理任务）

```mermaid
graph TB
subgraph "应用层"
APP["crm-app<br/>启动类与全局配置"]
end
subgraph "认证授权"
AUTH["crm-auth<br/>控制器/安全/服务/Mapper"]
end
subgraph "基础能力"
BASE["crm-base<br/>异常/结果/过滤器/配置"]
end
subgraph "文件服务"
FILE["crm-file<br/>控制器/服务/存储/定时任务"]
end
APP --> AUTH
APP --> FILE
AUTH --> BASE
FILE --> BASE
```

**图示来源** 
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
- [application.yml](file://crm-app/src/main/resources/application.yml)

**章节来源**
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
- [application.yml](file://crm-app/src/main/resources/application.yml)

## 核心组件
- 认证与鉴权
  - 登录接口与令牌签发校验：AuthController、TokenService、JwtAuthenticationFilter
  - 安全配置与异常处理：SecurityConfig、SecurityExceptionHandlers
  - 用户与权限服务：AuthUserServiceImpl、PermissionServiceImpl
  - 数据范围控制：DataScopeInterceptor
- 基础能力
  - 全局异常处理：GlobalExceptionHandlerAdvice
  - 统一错误码与业务异常：ResultCodeEnum、BusinessErrorException、PermissionErrorException
  - 请求链路追踪：TraceIdFilter
- 文件服务
  - 文件上传/下载/分片：FileController、FileApiImpl、FileInfoServiceImpl
  - MinIO 存储配置：MinioConfig
  - 分片残留清理：OrphanChunkCleanupTask

**章节来源**
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [SecurityConfig.java](file://crm-auth/src/main/java/com/crm/auth/config/SecurityConfig.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [SecurityExceptionHandlers.java](file://crm-auth/src/main/java/com/crm/auth/security/SecurityExceptionHandlers.java)
- [AuthUserServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/AuthUserServiceImpl.java)
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [DataScopeInterceptor.java](file://crm-auth/src/main/java/com/crm/auth/security/DataScopeInterceptor.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
- [PermissionErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/PermissionErrorException.java)
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
- [TraceIdFilter.java](file://crm-base/src/main/java/com/crm/base/filter/TraceIdFilter.java)
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)

## 架构总览
下图展示一次“登录+鉴权”的端到端调用流程，以及关键异常与错误码的流转路径。

```mermaid
sequenceDiagram
participant Client as "客户端"
participant AuthCtrl as "AuthController"
participant TokenSvc as "TokenService"
participant UserSvc as "AuthUserServiceImpl"
participant PermSvc as "PermissionServiceImpl"
participant Filter as "JwtAuthenticationFilter"
participant BaseEx as "GlobalExceptionHandlerAdvice"
Client->>AuthCtrl : "POST /auth/login"
AuthCtrl->>UserSvc : "验证用户名/密码"
UserSvc-->>AuthCtrl : "用户信息或错误"
AuthCtrl->>TokenSvc : "签发JWT"
TokenSvc-->>AuthCtrl : "返回令牌"
AuthCtrl-->>Client : "登录成功响应"
Client->>Filter : "携带Token访问受保护接口"
Filter->>TokenSvc : "解析并校验Token"
TokenSvc-->>Filter : "有效/无效"
Filter->>PermSvc : "权限/数据范围校验"
PermSvc-->>Filter : "通过/拒绝"
Filter-->>Client : "响应或403/401"
Note over BaseEx,Filter : "异常由全局处理器统一捕获并返回标准错误码"
```

**图示来源** 
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [AuthUserServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/AuthUserServiceImpl.java)
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)

## 详细组件分析

### 认证与鉴权组件
- 登录流程
  - 控制器接收登录参数，调用用户服务进行凭证校验，成功后由令牌服务签发JWT。
  - 后续请求经JWT过滤器解析令牌，再交由权限服务进行方法级与数据范围校验。
- 常见错误点
  - 令牌过期、签名不匹配、缺失Header
  - 角色/菜单未绑定、数据范围不足导致403
  - 用户不存在、密码错误、账号锁定

```mermaid
classDiagram
class AuthController {
+登录()
}
class TokenService {
+签发()
+校验()
}
class JwtAuthenticationFilter {
+过滤()
}
class PermissionServiceImpl {
+鉴权()
}
class GlobalExceptionHandlerAdvice {
+统一异常处理()
}
AuthController --> TokenService : "使用"
JwtAuthenticationFilter --> TokenService : "校验"
JwtAuthenticationFilter --> PermissionServiceImpl : "鉴权"
GlobalExceptionHandlerAdvice <.. JwtAuthenticationFilter : "捕获异常"
```

**图示来源** 
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)

**章节来源**
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)

### 文件服务组件
- 上传流程
  - 客户端发起上传请求，控制器调用文件API实现，底层持久化元数据至数据库，实际内容落盘到MinIO。
  - 支持分片上传与会话管理，定时任务清理孤立分片。
- 常见错误点
  - MinIO连接失败、Bucket不存在、网络超时
  - 分片大小限制、并发冲突、会话丢失
  - 权限不足导致无法读取/写入对象

```mermaid
flowchart TD
Start(["开始"]) --> CheckCfg["检查MinIO配置"]
CheckCfg --> Conn{"连接成功?"}
Conn --> |否| ErrConn["记录错误并返回连接失败"]
Conn --> |是| Upload["执行上传/分片上传"]
Upload --> Meta["持久化文件元数据"]
Meta --> Done(["完成"])
ErrConn --> End(["结束"])
Done --> End
```

**图示来源** 
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)

**章节来源**
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)

## 依赖关系分析
- 模块间依赖
  - crm-app 依赖 crm-auth、crm-file
  - crm-auth、crm-file 均依赖 crm-base（统一异常、结果、过滤器、配置）
- 运行时依赖
  - JWT 令牌校验链：过滤器 → 令牌服务 → 权限服务
  - 文件服务：控制器 → 文件API → 元数据服务 → MinIO

```mermaid
graph LR
APP["crm-app"] --> AUTH["crm-auth"]
APP --> FILE["crm-file"]
AUTH --> BASE["crm-base"]
FILE --> BASE
```

**图示来源** 
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
- [application.yml](file://crm-app/src/main/resources/application.yml)

**章节来源**
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
- [application.yml](file://crm-app/src/main/resources/application.yml)

## 性能与稳定性
- 认证鉴权
  - JWT 解析与签名校验为CPU密集型，需关注CPU峰值与GC停顿
  - 权限计算可能涉及多次DB查询，建议缓存热点数据（如菜单、角色）
- 文件服务
  - 大文件上传易触发内存峰值，应启用流式处理与分片策略
  - MinIO连接池与超时设置影响吞吐与稳定性
- 数据库
  - 连接池耗尽会导致请求阻塞，需监控活跃连接、等待队列与慢SQL
- 线程模型
  - 避免在过滤器/拦截器中执行耗时操作
  - 定时任务应避免阻塞主线程

[本节为通用指导，无需特定文件引用]

## 故障排查指南

### 认证失败（401/登录失败）
- 症状
  - 登录接口返回错误；后续接口报未认证或令牌无效
- 排查要点
  - 检查登录参数是否正确（用户名、密码、验证码等）
  - 确认JWT签发与校验配置一致（密钥、算法、过期时间）
  - 查看请求是否携带Authorization头及格式是否正确
  - 核对用户状态（是否存在、是否被禁用）
- 定位步骤
  - 通过TraceIdFilter生成的链路ID，聚合日志定位登录与鉴权链路
  - 查看全局异常处理器输出的错误码与消息
  - 检查TokenService与JwtAuthenticationFilter的日志输出
- 修复建议
  - 修正客户端请求头与参数
  - 调整JWT配置（密钥、过期策略）
  - 修复用户数据或状态

**章节来源**
- [AuthController.java](file://crm-auth/src/main/java/com/crm/auth/controller/AuthController.java)
- [JwtAuthenticationFilter.java](file://crm-auth/src/main/java/com/crm/auth/security/JwtAuthenticationFilter.java)
- [TokenService.java](file://crm-auth/src/main/java/com/crm/auth/security/TokenService.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)

### 权限拒绝（403/数据范围不足）
- 症状
  - 接口返回权限不足或无数据权限
- 排查要点
  - 用户角色与菜单/按钮权限是否绑定
  - 数据范围注解与上下文是否正确注入
  - 是否跨部门访问受限
- 定位步骤
  - 查看权限服务的鉴权逻辑与返回的错误码
  - 检查数据范围拦截器的SQL拼接与上下文变量
  - 结合TraceIdFilter链路ID检索相关日志
- 修复建议
  - 补全角色-菜单关联与用户-角色关联
  - 调整数据范围策略或用户所属部门
  - 优化权限缓存命中率

**章节来源**
- [PermissionServiceImpl.java](file://crm-auth/src/main/java/com/crm/auth/service/impl/PermissionServiceImpl.java)
- [DataScopeInterceptor.java](file://crm-auth/src/main/java/com/crm/auth/security/DataScopeInterceptor.java)
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
- [PermissionErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/PermissionErrorException.java)

### 文件上传失败
- 症状
  - 上传接口报错、分片上传中断、预览失败
- 排查要点
  - MinIO配置（地址、端口、AK/SK、Bucket）是否正确
  - 网络连通性与防火墙策略
  - 分片大小、并发数、超时时间设置
  - 存储空间配额与对象命名冲突
- 定位步骤
  - 查看MinioConfig与FileApiImpl的日志
  - 检查FileInfoServiceImpl的元数据持久化是否成功
  - 观察OrphanChunkCleanupTask是否清理了孤立分片
- 修复建议
  - 修正MinIO连接参数与网络策略
  - 调整分片策略与重试机制
  - 增加存储容量与监控告警

**章节来源**
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)

### 数据库连接异常
- 症状
  - 启动时报连接失败、运行中频繁出现连接超时或获取连接超时
- 排查要点
  - 数据库地址、端口、用户名、密码、SSL配置
  - 连接池大小、最大空闲、超时时间
  - 慢SQL与长事务导致连接占用
- 定位步骤
  - 查看应用日志中的连接池与SQL异常堆栈
  - 使用数据库监控工具查看活跃会话与锁等待
  - 开启慢查询日志，定位热点SQL
- 修复建议
  - 修正连接参数与网络策略
  - 调整连接池参数与SQL索引
  - 拆分长事务与批量操作

**章节来源**
- [application.yml](file://crm-auth/src/main/resources/application.yml)
- [application.yml](file://crm-app/src/main/resources/application.yml)

### 日志分析与链路追踪
- 使用TraceIdFilter生成链路ID，贯穿请求生命周期
- 将链路ID纳入日志关键字段，便于聚合检索
- 针对认证、权限、文件、数据库等关键路径打印必要上下文

**章节来源**
- [TraceIdFilter.java](file://crm-base/src/main/java/com/crm/base/filter/TraceIdFilter.java)

### 全局异常与错误码
- 统一异常处理器捕获业务异常与系统异常，返回标准结果
- 错误码枚举集中管理，便于前端与监控对接

**章节来源**
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
- [PermissionErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/PermissionErrorException.java)
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)

## 结论
通过模块化设计与统一的异常、日志、鉴权体系，系统在可观测性与可维护性方面具备良好基础。围绕认证、权限、文件、数据库四大类常见问题，结合链路追踪与监控手段，能够快速定位与修复问题，显著降低停机风险。

[本节为总结性内容，无需特定文件引用]

## 附录：常用命令与工具
- JVM与进程
  - jps、jstack、jmap、jstat、jcmd
  - 线程dump分析：查找阻塞与死锁
  - 内存快照分析：定位对象增长与泄漏
- 网络与连接
  - netstat、ss、tcpdump、curl、wget
  - MinIO客户端：mc stat、mc ls、mc cp
- 数据库
  - 连接池监控：活跃连接、等待队列、慢SQL
  - 慢查询日志：开启与分析
- 日志与监控
  - 基于TraceIdFilter的链路ID检索
  - 统一错误码与异常信息的聚合统计
  - 关键指标：QPS、延迟分布、错误率、连接池利用率、GC停顿

[本节为通用指导，无需特定文件引用]