You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
502 lines
25 KiB
502 lines
25 KiB
|
1 month ago
|
# 文件服务
|
||
|
|
|
||
|
|
<cite>
|
||
|
|
**本文引用的文件**
|
||
|
|
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
|
||
|
|
- [application.yml](file://crm-app/src/main/resources/application.yml)
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApi.java](file://crm-file/src/main/java/com/crm/file/api/FileApi.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [IFileInfoService.java](file://crm-file/src/main/java/com/crm/file/service/IFileInfoService.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
- [FileDownloadDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileDownloadDTO.java)
|
||
|
|
- [FileConstants.java](file://crm-file/src/main/java/com/crm/file/constant/FileConstants.java)
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
|
||
|
|
- [RedisConfig.java](file://crm-base/src/main/java/com/crm/base/config/RedisConfig.java)
|
||
|
|
</cite>
|
||
|
|
|
||
|
|
## 目录
|
||
|
|
1. [简介](#简介)
|
||
|
|
2. [项目结构](#项目结构)
|
||
|
|
3. [核心组件](#核心组件)
|
||
|
|
4. [架构总览](#架构总览)
|
||
|
|
5. [详细组件分析](#详细组件分析)
|
||
|
|
6. [依赖关系分析](#依赖关系分析)
|
||
|
|
7. [性能考虑](#性能考虑)
|
||
|
|
8. [故障排查指南](#故障排查指南)
|
||
|
|
9. [结论](#结论)
|
||
|
|
10. [附录](#附录)
|
||
|
|
|
||
|
|
## 简介
|
||
|
|
本技术文档面向“文件服务”模块,覆盖文件上传下载、分片上传、断点续传、文件预览、批量操作等能力;深入解析 MinIO 对象存储集成、文件元数据管理、异步任务处理与清理调度;提供 API 接口规范、错误处理机制、性能优化策略;并给出配置指南、客户端集成示例与最佳实践。同时阐述文件生命周期管理与存储空间优化方案,帮助读者快速理解与高效使用。
|
||
|
|
|
||
|
|
## 项目结构
|
||
|
|
文件服务位于 crm-file 模块,采用分层架构:
|
||
|
|
- 控制器层:对外暴露 REST 接口(上传、下载、预览、分片、批量等)
|
||
|
|
- 服务层:封装业务逻辑(文件信息、分片会话、MinIO 操作、预览调用)
|
||
|
|
- 配置层:MinIO、文件属性、定时任务调度
|
||
|
|
- 领域模型:实体、DTO、常量
|
||
|
|
- 持久化:MyBatis-Plus Mapper
|
||
|
|
- 任务:清理孤儿分片的定时任务
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
graph TB
|
||
|
|
subgraph "crm-file"
|
||
|
|
FC["FileController"] --> FAI["FileApiImpl"]
|
||
|
|
FAI --> FMS["FileInfoServiceImpl"]
|
||
|
|
FAI --> MINIO["MinioConfig(客户端)"]
|
||
|
|
FAI --> KK["KkFileViewClient"]
|
||
|
|
FMS --> FIM["FileInfoMapper"]
|
||
|
|
TASK["OrphanChunkCleanupTask"] --> FMS
|
||
|
|
CFG["FileProperties / SchedulingConfig"] --> TASK
|
||
|
|
MINCFG["MinioConfig"] --> MINIO
|
||
|
|
end
|
||
|
|
subgraph "crm-base"
|
||
|
|
EXC["GlobalExceptionHandlerAdvice"]
|
||
|
|
RES["Result / ResultCodeEnum"]
|
||
|
|
REDIS["RedisConfig"]
|
||
|
|
end
|
||
|
|
FC --> EXC
|
||
|
|
FC --> RES
|
||
|
|
FMS --> REDIS
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [RedisConfig.java](file://crm-base/src/main/java/com/crm/base/config/RedisConfig.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [RedisConfig.java](file://crm-base/src/main/java/com/crm/base/config/RedisConfig.java)
|
||
|
|
|
||
|
|
## 核心组件
|
||
|
|
- FileController:统一入口,定义上传、下载、预览、分片初始化、分片上传、合并、批量删除等接口
|
||
|
|
- FileApiImpl:文件业务编排,协调 MinIO、元数据、预览、分片会话
|
||
|
|
- FileInfoServiceImpl:文件元数据 CRUD、查询、统计、生命周期相关操作
|
||
|
|
- MinioConfig:MinIO 客户端配置与连接参数
|
||
|
|
- KkFileViewClient:第三方文件预览服务客户端
|
||
|
|
- OrphanChunkCleanupTask:定时清理未完成的分片上传残留
|
||
|
|
- FileProperties/SchedulingConfig:文件服务与调度任务配置
|
||
|
|
- FileInfo/FileInfoMapper:元数据实体与持久化映射
|
||
|
|
- DTO:分片初始化、上传会话、文件信息等数据传输对象
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
- [FileDownloadDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileDownloadDTO.java)
|
||
|
|
|
||
|
|
## 架构总览
|
||
|
|
文件服务以 Controller 为入口,通过 Service 编排 MinIO 对象存储与预览服务,元数据落库,分片会话与进度由 Redis 支撑,定时任务负责清理与生命周期管理。
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
sequenceDiagram
|
||
|
|
participant Client as "客户端"
|
||
|
|
participant Ctrl as "FileController"
|
||
|
|
participant Api as "FileApiImpl"
|
||
|
|
participant Meta as "FileInfoServiceImpl"
|
||
|
|
participant MinIO as "MinIO 客户端"
|
||
|
|
participant Preview as "KkFileViewClient"
|
||
|
|
participant DB as "数据库"
|
||
|
|
participant Cache as "Redis"
|
||
|
|
Client->>Ctrl : "POST /api/file/upload/init"
|
||
|
|
Ctrl->>Api : "initMultipart(params)"
|
||
|
|
Api->>MinIO : "创建分片上传任务"
|
||
|
|
Api->>Cache : "保存 UploadSession"
|
||
|
|
Api-->>Ctrl : "返回 uploadId/session"
|
||
|
|
Ctrl-->>Client : "响应"
|
||
|
|
Client->>Ctrl : "POST /api/file/chunk"
|
||
|
|
Ctrl->>Api : "uploadChunk(uploadId, chunkIndex, data)"
|
||
|
|
Api->>MinIO : "上传分片"
|
||
|
|
Api->>Cache : "更新进度/状态"
|
||
|
|
Api-->>Ctrl : "成功"
|
||
|
|
Ctrl-->>Client : "响应"
|
||
|
|
Client->>Ctrl : "POST /api/file/merge"
|
||
|
|
Ctrl->>Api : "completeMultipart(uploadId)"
|
||
|
|
Api->>MinIO : "合并分片"
|
||
|
|
Api->>Meta : "写入 FileInfo 元数据"
|
||
|
|
Meta->>DB : "INSERT/UPDATE"
|
||
|
|
Api-->>Ctrl : "返回文件信息"
|
||
|
|
Ctrl-->>Client : "响应"
|
||
|
|
Client->>Ctrl : "GET /api/file/download/{id}"
|
||
|
|
Ctrl->>Api : "download(id)"
|
||
|
|
Api->>Meta : "查询元数据"
|
||
|
|
Meta->>DB : "SELECT"
|
||
|
|
Api->>MinIO : "GetObject"
|
||
|
|
Api-->>Ctrl : "流式返回"
|
||
|
|
Ctrl-->>Client : "文件流"
|
||
|
|
Client->>Ctrl : "GET /api/file/preview/{id}"
|
||
|
|
Ctrl->>Api : "preview(id)"
|
||
|
|
Api->>Preview : "生成预览链接/页面"
|
||
|
|
Preview-->>Api : "预览地址"
|
||
|
|
Api-->>Ctrl : "返回预览URL"
|
||
|
|
Ctrl-->>Client : "跳转预览"
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [RedisConfig.java](file://crm-base/src/main/java/com/crm/base/config/RedisConfig.java)
|
||
|
|
|
||
|
|
## 详细组件分析
|
||
|
|
|
||
|
|
### 文件上传下载与分片上传
|
||
|
|
- 分片初始化:生成唯一 uploadId,建立 UploadSession,记录分片大小、文件名、类型、MD5 等
|
||
|
|
- 分片上传:按索引顺序或乱序上传,服务端校验分片完整性,更新 Redis 中的进度
|
||
|
|
- 合并分片:校验所有分片完成,调用 MinIO 合并,成功后写入元数据并清理会话
|
||
|
|
- 下载:根据文件 ID 查询元数据,从 MinIO 拉取流式响应,支持范围请求与缓存头
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
flowchart TD
|
||
|
|
Start(["开始"]) --> Init["初始化分片<br/>生成 uploadId/会话"]
|
||
|
|
Init --> UploadLoop{"是否还有分片?"}
|
||
|
|
UploadLoop --> |是| Chunk["上传分片<br/>校验/更新进度"]
|
||
|
|
Chunk --> UploadLoop
|
||
|
|
UploadLoop --> |否| Merge["合并分片<br/>校验完整性"]
|
||
|
|
Merge --> SaveMeta["保存元数据<br/>写入数据库"]
|
||
|
|
SaveMeta --> Clean["清理会话/临时数据"]
|
||
|
|
Clean --> End(["结束"])
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
|
||
|
|
### 断点续传与会话管理
|
||
|
|
- 会话存储:UploadSession 保存在 Redis,包含 uploadId、分片列表、状态、过期时间
|
||
|
|
- 断点恢复:客户端携带 lastChunkIndex,服务端跳过已上传分片,继续后续上传
|
||
|
|
- 超时清理:结合定时任务清理长时间未完成会话,避免资源泄漏
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
classDiagram
|
||
|
|
class UploadSession {
|
||
|
|
+string uploadId
|
||
|
|
+string fileName
|
||
|
|
+long fileSize
|
||
|
|
+int chunkSize
|
||
|
|
+String[] uploadedChunks
|
||
|
|
+int totalChunks
|
||
|
|
+string status
|
||
|
|
+datetime createdAt
|
||
|
|
+datetime expiredAt
|
||
|
|
}
|
||
|
|
class MultipartInitDTO {
|
||
|
|
+string fileName
|
||
|
|
+long fileSize
|
||
|
|
+string contentType
|
||
|
|
+int chunkSize
|
||
|
|
}
|
||
|
|
UploadSession <.. MultipartInitDTO : "由初始化参数构建"
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
|
||
|
|
### 文件预览
|
||
|
|
- 预览服务:通过 KkFileViewClient 调用外部预览服务,返回可访问的预览 URL
|
||
|
|
- 安全控制:仅允许已授权用户访问,必要时增加鉴权参数或签名
|
||
|
|
- 缓存策略:对热门文件可缓存预览结果或缩短 TTL
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
sequenceDiagram
|
||
|
|
participant C as "客户端"
|
||
|
|
participant Ctrl as "FileController"
|
||
|
|
participant Api as "FileApiImpl"
|
||
|
|
participant KV as "KkFileViewClient"
|
||
|
|
C->>Ctrl : "GET /api/file/preview/{id}"
|
||
|
|
Ctrl->>Api : "preview(id)"
|
||
|
|
Api->>KV : "请求预览链接"
|
||
|
|
KV-->>Api : "返回预览URL"
|
||
|
|
Api-->>Ctrl : "返回URL"
|
||
|
|
Ctrl-->>C : "重定向到预览页"
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
|
||
|
|
### 批量操作
|
||
|
|
- 批量删除:传入多个文件 ID,逐个校验权限与存在性,执行删除并返回结果集
|
||
|
|
- 批量导出:基于元数据查询导出 Excel,适合报表场景
|
||
|
|
- 并发控制:限制并发度,避免瞬时压力过大
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
|
||
|
|
### MinIO 对象存储集成
|
||
|
|
- 连接配置:端点、AccessKey、SecretKey、Bucket、区域、SSL 等
|
||
|
|
- 上传下载:分片上传、合并、流式下载、范围请求
|
||
|
|
- 桶策略:读写权限、生命周期规则、版本控制(可选)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
|
||
|
|
### 文件元数据管理
|
||
|
|
- 实体字段:文件名、路径、大小、类型、MD5、上传者、创建时间、状态等
|
||
|
|
- 持久化:MyBatis-Plus Mapper 实现增删改查与条件查询
|
||
|
|
- 审计:记录操作人、IP、来源设备等信息(可扩展)
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
erDiagram
|
||
|
|
FILE_INFO {
|
||
|
|
bigint id PK
|
||
|
|
string file_name
|
||
|
|
string file_path
|
||
|
|
long file_size
|
||
|
|
string content_type
|
||
|
|
string md5
|
||
|
|
string uploader_id
|
||
|
|
datetime created_at
|
||
|
|
datetime updated_at
|
||
|
|
enum status
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
|
||
|
|
### 异步任务与清理调度
|
||
|
|
- 孤儿分片清理:扫描超时的 UploadSession,删除 MinIO 中未完成分片,释放空间
|
||
|
|
- 生命周期管理:定期归档或删除过期文件,降低存储成本
|
||
|
|
- 调度配置:基于 Spring Scheduler,支持 Cron 表达式与线程池隔离
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
flowchart TD
|
||
|
|
TStart["定时触发"] --> Scan["扫描过期会话"]
|
||
|
|
Scan --> ForEach{"遍历会话"}
|
||
|
|
ForEach --> |存在| Cleanup["删除分片/清理Redis"]
|
||
|
|
ForEach --> |不存在| Next["下一个"]
|
||
|
|
Cleanup --> Next
|
||
|
|
Next --> TEnd["结束"]
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
|
||
|
|
## 依赖关系分析
|
||
|
|
- 控制器依赖服务接口,服务实现依赖 MinIO 客户端、Redis、数据库
|
||
|
|
- 预览功能依赖外部 KkFileView 服务
|
||
|
|
- 定时任务依赖 Redis 与会话状态,以及 MinIO 的分片管理能力
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
graph LR
|
||
|
|
FC["FileController"] --> FAI["FileApiImpl"]
|
||
|
|
FAI --> FMS["FileInfoServiceImpl"]
|
||
|
|
FAI --> MIN["MinioConfig"]
|
||
|
|
FAI --> KK["KkFileViewClient"]
|
||
|
|
FMS --> FIM["FileInfoMapper"]
|
||
|
|
TASK["OrphanChunkCleanupTask"] --> FMS
|
||
|
|
TASK --> MIN
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
|
||
|
|
## 性能考虑
|
||
|
|
- 分片大小与并发:合理设置分片大小(如 5MB),提升网络吞吐与重试效率
|
||
|
|
- 流式传输:下载使用流式响应,避免内存峰值
|
||
|
|
- 缓存预热:热点文件在 Redis 缓存元数据或预览链接,减少后端压力
|
||
|
|
- 连接池:MinIO 客户端连接池与 HTTP 客户端连接池调优
|
||
|
|
- 异步处理:大文件处理与预览生成走异步队列,避免阻塞主流程
|
||
|
|
- 压缩与转码:按需启用压缩与格式转换,平衡质量与体积
|
||
|
|
|
||
|
|
[本节为通用指导,不直接分析具体文件]
|
||
|
|
|
||
|
|
## 故障排查指南
|
||
|
|
- 全局异常处理:统一捕获业务异常与系统异常,返回标准 Result 结构
|
||
|
|
- 常见错误:
|
||
|
|
- 分片缺失或顺序错误:检查 UploadSession 与 Redis 进度
|
||
|
|
- MinIO 连接失败:核对 MinioConfig 配置与网络连通性
|
||
|
|
- 预览服务不可用:检查 KkFileView 服务健康与超时配置
|
||
|
|
- 权限不足:校验用户身份与数据可见性
|
||
|
|
- 日志与追踪:开启 TraceId,定位跨服务调用链路
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
|
||
|
|
|
||
|
|
## 结论
|
||
|
|
文件服务围绕 MinIO 对象存储与 Redis 会话管理,构建了完整的上传下载、分片断点续传、预览与批量处理能力。通过清晰的层次划分与标准化接口,便于扩展与维护。建议在生产环境完善监控告警、限流熔断与审计合规,持续提升稳定性与性能。
|
||
|
|
|
||
|
|
[本节为总结,不直接分析具体文件]
|
||
|
|
|
||
|
|
## 附录
|
||
|
|
|
||
|
|
### API 接口规范(摘要)
|
||
|
|
- 上传初始化
|
||
|
|
- 方法:POST
|
||
|
|
- 路径:/api/file/upload/init
|
||
|
|
- 入参:文件名、大小、类型、分片大小
|
||
|
|
- 出参:uploadId、会话信息
|
||
|
|
- 分片上传
|
||
|
|
- 方法:POST
|
||
|
|
- 路径:/api/file/chunk
|
||
|
|
- 入参:uploadId、分片索引、分片数据
|
||
|
|
- 出参:上传状态
|
||
|
|
- 合并分片
|
||
|
|
- 方法:POST
|
||
|
|
- 路径:/api/file/merge
|
||
|
|
- 入参:uploadId
|
||
|
|
- 出参:文件信息
|
||
|
|
- 下载
|
||
|
|
- 方法:GET
|
||
|
|
- 路径:/api/file/download/{id}
|
||
|
|
- 出参:文件流
|
||
|
|
- 预览
|
||
|
|
- 方法:GET
|
||
|
|
- 路径:/api/file/preview/{id}
|
||
|
|
- 出参:预览 URL
|
||
|
|
- 批量删除
|
||
|
|
- 方法:POST
|
||
|
|
- 路径:/api/file/batch/delete
|
||
|
|
- 入参:文件ID列表
|
||
|
|
- 出参:操作结果
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
|
||
|
|
### 配置指南
|
||
|
|
- MinIO 配置
|
||
|
|
- 端点、AccessKey、SecretKey、Bucket、区域、SSL
|
||
|
|
- 文件服务配置
|
||
|
|
- 默认分片大小、最大文件大小、保留天数、清理周期
|
||
|
|
- 调度任务配置
|
||
|
|
- Cron 表达式、线程池大小、超时时间
|
||
|
|
- Redis 配置
|
||
|
|
- 连接信息、序列化方式、键前缀、TTL
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [RedisConfig.java](file://crm-base/src/main/java/com/crm/base/config/RedisConfig.java)
|
||
|
|
|
||
|
|
### 客户端集成示例(步骤)
|
||
|
|
- 初始化分片:获取 uploadId
|
||
|
|
- 循环上传分片:按索引上传,记录成功数
|
||
|
|
- 合并分片:完成后合并,等待元数据写入
|
||
|
|
- 下载与预览:根据文件 ID 下载或打开预览链接
|
||
|
|
- 错误重试:针对网络抖动进行指数退避重试
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
|
||
|
|
### 最佳实践
|
||
|
|
- 分片大小与并发:根据网络与服务器能力调优
|
||
|
|
- 幂等设计:上传与合并接口具备幂等性,避免重复提交
|
||
|
|
- 安全校验:严格校验文件类型、大小、MD5,防止恶意上传
|
||
|
|
- 监控告警:关键指标(成功率、延迟、存储用量)接入监控
|
||
|
|
- 容量规划:冷热分层、生命周期策略、压缩归档
|
||
|
|
|
||
|
|
[本节为通用指导,不直接分析具体文件]
|
||
|
|
|
||
|
|
### 文件生命周期与存储优化
|
||
|
|
- 生命周期策略:按时间或标签自动归档/删除
|
||
|
|
- 版本控制:开启版本控制以便回滚与审计
|
||
|
|
- 冷热分层:热数据 SSD,冷数据低成本介质
|
||
|
|
- 去重与压缩:相同内容只存一份,压缩静态资源
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|