TIP
研发工作中遇到过最为抓狂的事情可能就是排查线上问题了。明显易发现的问题还好，但有些问题只在生产环境的高并发真实流量场景中才会出现，相同的分支代码放到线下又可以正常运行了，每当此时也只能以问题很难复现再观察定位的话术来敷衍。那么生产环境问题为什么难定位呢：
🟢 生产环境不能debug，看不到流程的全貌，很多时候流程真正的执行链路需要靠猜测
🟢 虽然可以通过日志查看到很多细节和问题，但日志又不能无限制的使用，不然即影响了开发效率，又影响了代码的可读性，日志加的太多还容易造成生产环境机器的计算资源、存储资源浪费
🟢 一些极端场景的链路或数据，只有生产环境才会出现
为了减少问题排查的复杂程度，Kstry 提供了流程回溯功能，流程回溯记录了节点的信息、执行顺序、耗时、入参、出参等重要数据
# 6.1 traceId支持
Kstry 内部使用MDC工具，支持设置全链路请求ID
🟢 StoryEngine.fire入参中有requestId属性可以指定当前请求的唯一ID，如果未指定引擎会使用UUID生成一个
🟢 日志打印时可以获取 traceId 来串联当前请求的全部日志，traceId 默认名是：ks-request-id
<property name="FILE_LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread][%X{ks-request-id}] %-5level %logger{20} - %msg%n"/>
1
🟢 traceId 默认名可以修改：
# application.yml
kstry:
story:
request-id: traceId
1
2
3
4
2
3
4
# 6.2 流程回溯
# 6.2.1 流程回溯类型
Kstry 支持五种模式的回溯
🟢 NONE：不开启回溯功能
🟢 ALL：开启全部采集点的回溯，包括所有bpmn元素执行，节点信息，节点入参，节点出参，耗时等
🟢 NODE：只采集bpmn节点执行流程，不关注节点执行细节
🟢 SERVICE：只关注bpmn元素中的 ServiceTask 节点，也就是服务节点的执行，采集节点的信息、耗时、执行顺序等，但是不采集出参、入参
🟢 SERVICE_DETAIL：在SERVICE的基础上增加了对出参、入参的采集
# 6.2.2 流程回溯使用
执行入参中显示指定：
@RestController
@RequestMapping("/goods")
public class GoodsController {
@Resource
private StoryEngine storyEngine;
@PostMapping("/show")
public GoodsDetail showGoods(@RequestBody GoodsDetailRequest request) {
StoryRequest<GoodsDetail> req = ReqBuilder.returnType(GoodsDetail.class).startId("kstry-demo-goods-show").trackingType(TrackingTypeEnum.SERVICE).request(request).build();
TaskResponse<GoodsDetail> fire = storyEngine.fire(req);
if (fire.isSuccess()) {
return fire.getResult();
}
return null;
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
全局默认配置：
# application.yml
kstry:
story:
tracking:
type: service
log: true # 是否打印回溯log，默认为 false
1
2
3
4
5
6
7
2
3
4
5
6
7
# 6.2.3 回溯日志样例
🟢 ALL：
[K1040009] startId: kstry-demo-goods-show, spend 631ms: [{
"index": 1,
"nodeId": "kstry-demo-goods-show",
"nodeName": "kstry-demo-goods-show",
"nodeType": "START_EVENT",
"noticeTracking": [],
"paramTracking": [],
"startTime": "2021-12-17T01:36:20.926",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Flow_1uuw3lx"]
}, {
"index": 2,
"nodeId": "Flow_1uuw3lx",
"nodeType": "SEQUENCE_FLOW",
"noticeTracking": [],
"paramTracking": [],
"startTime": "2021-12-17T01:36:20.926",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Activity_1t7q7ro"]
}, {
"endTime": "2021-12-17T01:36:20.993",
"index": 3,
"methodName": "initBaseInfo",
"nodeId": "Activity_1t7q7ro",
"nodeName": "初始化\n基本信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [{
"fieldName": "goodsDetail",
"noticeName": "goodsDetail",
"noticeScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}, {
"noticeScopeType": "RESULT",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}],
"paramTracking": [{
"paramName": "request",
"sourceName": "request",
"sourceScopeType": "REQUEST",
"value": "{\"businessId\":\"special-channel\",\"id\":1,\"source\":\"app\"}"
}, {
"paramName": "goodsDetail",
"sourceName": "goodsDetail",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}, {
"paramName": "role",
"sourceName": "role",
"sourceScopeType": "EMPTY",
"value": "null"
}],
"spendTime": 67,
"startTime": "2021-12-17T01:36:20.926",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Flow_0dt6q7w"]
}
......
{
"endTime": "2021-12-17T01:36:21.549",
"index": 46,
"methodName": "detailPostProcess",
"nodeId": "Activity_1s51gmf",
"nodeName": "商详后置处理",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [{
"paramName": "request.shopInfo",
"sourceName": "shopInfo",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"labels\":[{\"index\":1,\"name\":\"L1\",\"url\":\"https://xxx.png\"},{\"index\":2,\"name\":\"L2\",\"url\":\"https://xxx2.png\"}],\"salesNumMonthly\":22,\"shopName\":\"店铺名称\"}"
}, {
"paramName": "request.goodsDetail",
"sourceName": "goodsDetail",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true,\"skuInfos\":[{\"img\":\"https://xxx.png\",\"price\":1000,\"skuId\":100,\"skuName\":\"sku1\",\"stock\":10},{\"img\":\"https://xxx2.png\",\"price\":2000,\"skuId\":101,\"skuName\":\"sku2\",\"stock\":20}]}"
}, {
"paramName": "request.logisticInsurance",
"sourceName": "logisticInsurance",
"sourceScopeType": "STABLE",
"value": "{\"desc\":\"运费险描述\",\"type\":1}"
}, {
"paramName": "request.evaluationInfo",
"sourceName": "evaluationInfo",
"sourceScopeType": "STABLE",
"value": "{\"evaluateCount\":20}"
}, {
"paramName": "request.orderInfo",
"sourceName": "orderInfo",
"sourceScopeType": "STABLE",
"value": "{\"orderedCount\":10}"
}, {
"paramName": "request.goodsExtInfo",
"sourceName": "goodsExtInfo",
"sourceScopeType": "STABLE",
"value": "{\"collectCount\":30}"
}],
"spendTime": 11,
"startTime": "2021-12-17T01:36:21.538",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-8",
"toNodeIds": ["Flow_0rl59u8"]
}, {
"index": 47,
"nodeId": "Flow_0rl59u8",
"nodeType": "SEQUENCE_FLOW",
"noticeTracking": [],
"paramTracking": [],
"startTime": "2021-12-17T01:36:21.549",
"threadId": "kstry-task-thread-pool-8",
"toNodeIds": ["Event_1yjag2c"]
}, {
"index": 48,
"nodeId": "Event_1yjag2c",
"nodeType": "END_EVENT",
"noticeTracking": [],
"paramTracking": [],
"startTime": "2021-12-17T01:36:21.550",
"threadId": "kstry-task-thread-pool-8",
"toNodeIds": []
}]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
🟢 NODE：
[K1040009] startId: kstry-demo-goods-show, spend 772ms:
[{
"index": 1,
"nodeId": "kstry-demo-goods-show",
"nodeName": "kstry-demo-goods-show",
"nodeType": "START_EVENT",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": ["Flow_1uuw3lx"]
}, {
"index": 2,
"nodeId": "Flow_1uuw3lx",
"nodeType": "SEQUENCE_FLOW",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": ["Activity_1t7q7ro"]
}, {
"index": 3,
"nodeId": "Activity_1t7q7ro",
"nodeName": "初始化\n基本信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": ["Flow_0dt6q7w"]
}
......
{
"index": 46,
"nodeId": "Activity_1s51gmf",
"nodeName": "商详后置处理",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": ["Flow_0rl59u8"]
}, {
"index": 47,
"nodeId": "Flow_0rl59u8",
"nodeType": "SEQUENCE_FLOW",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": ["Event_1yjag2c"]
}, {
"index": 48,
"nodeId": "Event_1yjag2c",
"nodeType": "END_EVENT",
"noticeTracking": [],
"paramTracking": [],
"toNodeIds": []
}]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
🟢 SERVICE：
[K1040009] startId: kstry-demo-goods-show, spend 325ms:
[{
"endTime": "2021-12-17T01:41:54.152",
"index": 3,
"methodName": "initBaseInfo",
"nodeId": "Activity_1t7q7ro",
"nodeName": "初始化\n基本信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"spendTime": 57,
"startTime": "2021-12-17T01:41:54.095",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Activity_1iu4r2e"]
}, {
"ability": "triple",
"endTime": "2021-12-17T01:41:54.190",
"index": 5,
"methodName": "tripleCheckImg",
"nodeId": "Activity_1iu4r2e",
"nodeName": "风控服务",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"spendTime": 33,
"startTime": "2021-12-17T01:41:54.157",
"targetName": "cn.kstry.demo.service.RiskControlService",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Activity_1nhha2f"]
}
......
{
"endTime": "2021-12-17T01:41:54.243",
"index": 42,
"methodName": "getShopInfoByGoodsId",
"nodeId": "Activity_019xur8",
"nodeName": "加载店铺信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"spendTime": 13,
"startTime": "2021-12-17T01:41:54.230",
"targetName": "cn.kstry.demo.service.ShopService",
"threadId": "kstry-task-thread-pool-2",
"toNodeIds": ["Activity_1s51gmf"]
}, {
"endTime": "2021-12-17T01:41:54.409",
"index": 46,
"methodName": "detailPostProcess",
"nodeId": "Activity_1s51gmf",
"nodeName": "商详后置处理",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [],
"spendTime": 10,
"startTime": "2021-12-17T01:41:54.399",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-8",
"toNodeIds": []
}]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
🟢 SERVICE_DETAIL：
[K1040009] startId: kstry-demo-goods-show, spend 284ms:
[{
"endTime": "2021-12-17T01:43:43.177",
"index": 3,
"methodName": "initBaseInfo",
"nodeId": "Activity_1t7q7ro",
"nodeName": "初始化\n基本信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [{
"fieldName": "goodsDetail",
"noticeName": "goodsDetail",
"noticeScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}, {
"noticeScopeType": "RESULT",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}],
"paramTracking": [{
"paramName": "request",
"sourceName": "request",
"sourceScopeType": "REQUEST",
"value": "{\"businessId\":\"special-channel\",\"id\":1,\"source\":\"app\"}"
}, {
"paramName": "goodsDetail",
"sourceName": "goodsDetail",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true}"
}, {
"paramName": "role",
"sourceName": "role",
"sourceScopeType": "EMPTY",
"value": "null"
}],
"spendTime": 57,
"startTime": "2021-12-17T01:43:43.120",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Activity_1iu4r2e"]
}, {
"ability": "triple",
"endTime": "2021-12-17T01:43:43.210",
"index": 5,
"methodName": "tripleCheckImg",
"nodeId": "Activity_1iu4r2e",
"nodeName": "风控服务",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [{
"paramName": "checkInfo.img",
"sourceName": "goodsDetail.img",
"sourceScopeType": "STABLE",
"value": "https://xxx.png"
}],
"spendTime": 29,
"startTime": "2021-12-17T01:43:43.181",
"targetName": "cn.kstry.demo.service.RiskControlService",
"threadId": "kstry-task-thread-pool-0",
"toNodeIds": ["Activity_1nhha2f"]
}
......
{
"endTime": "2021-12-17T01:43:43.260",
"index": 42,
"methodName": "getShopInfoByGoodsId",
"nodeId": "Activity_019xur8",
"nodeName": "加载店铺信息",
"nodeType": "SERVICE_TASK",
"noticeTracking": [{
"fieldName": "shopInfo",
"noticeName": "shopInfo",
"noticeScopeType": "STABLE",
"value": "{\"id\":1,\"salesNumMonthly\":22,\"shopName\":\"店铺名称\"}"
}],
"paramTracking": [{
"paramName": "goodsId",
"sourceName": "id",
"sourceScopeType": "REQUEST",
"value": "1"
}],
"spendTime": 13,
"startTime": "2021-12-17T01:43:43.247",
"targetName": "cn.kstry.demo.service.ShopService",
"threadId": "kstry-task-thread-pool-2",
"toNodeIds": ["Activity_1s51gmf"]
}, {
"endTime": "2021-12-17T01:43:43.394",
"index": 46,
"methodName": "detailPostProcess",
"nodeId": "Activity_1s51gmf",
"nodeName": "商详后置处理",
"nodeType": "SERVICE_TASK",
"noticeTracking": [],
"paramTracking": [{
"paramName": "request.shopInfo",
"sourceName": "shopInfo",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"labels\":[{\"index\":1,\"name\":\"L1\",\"url\":\"https://xxx.png\"},{\"index\":2,\"name\":\"L2\",\"url\":\"https://xxx2.png\"}],\"salesNumMonthly\":22,\"shopName\":\"店铺名称\"}"
}, {
"paramName": "request.goodsDetail",
"sourceName": "goodsDetail",
"sourceScopeType": "STABLE",
"value": "{\"id\":1,\"img\":\"https://xxx.png\",\"name\":\"商品\",\"needEvaluate\":true,\"skuInfos\":[{\"img\":\"https://xxx.png\",\"price\":1000,\"skuId\":100,\"skuName\":\"sku1\",\"stock\":10},{\"img\":\"https://xxx2.png\",\"price\":2000,\"skuId\":101,\"skuName\":\"sku2\",\"stock\":20}]}"
}, {
"paramName": "request.logisticInsurance",
"sourceName": "logisticInsurance",
"sourceScopeType": "STABLE",
"value": "{\"desc\":\"运费险描述\",\"type\":1}"
}, {
"paramName": "request.evaluationInfo",
"sourceName": "evaluationInfo",
"sourceScopeType": "STABLE",
"value": "{\"evaluateCount\":20}"
}, {
"paramName": "request.orderInfo",
"sourceName": "orderInfo",
"sourceScopeType": "STABLE",
"value": "{\"orderedCount\":10}"
}, {
"paramName": "request.goodsExtInfo",
"sourceName": "goodsExtInfo",
"sourceScopeType": "STABLE",
"value": "{\"collectCount\":30}"
}],
"spendTime": 15,
"startTime": "2021-12-17T01:43:43.379",
"targetName": "cn.kstry.demo.service.GoodsService",
"threadId": "kstry-task-thread-pool-8",
"toNodeIds": []
}]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
# 6.3 节点状态事件
TIP
框架提供了TrackingBeginEvent、TrackingFinishEvent两个Spring容器事件，用来监听节点执行前后的日志对象，可以通过订阅Spring事件的方式使用
@Component
public class TrackingEventListener {
@EventListener
public void handleTrackingBeginEvent(TrackingBeginEvent event) {
System.out.println("TrackingBeginEvent：" + JSON.toJSONString(event));
}
@EventListener
public void handleTrackingFinishEvent(TrackingFinishEvent event) {
System.out.println("TrackingFinishEvent：" + JSON.toJSONString(event));
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
2
3
4
5
6
7
8
9
10
11
12
13
14
🟢 TrackingBeginEvent：节点开始执行前触发
🟢 TrackingFinishEvent：节点执行完成后触发
# 6.4 回溯自定义
回溯自定义是在链路执行完之后，拿到结果或者异常之前，执行的自定义回调方法，可以应对如下问题：
🟢 自定义流程回溯日志，甚至是出现异常时才打印回溯日志
🟢 检查节点执行，参数设置等是否符合预期。有时结果没有报错，并不代表一定是没有问题的
🟢 如果链路中有自定义角色的操作，检查最终角色是否符合预期
举例：
按时间维度的先后执行顺序，打印节点名及耗时：
@RestController
@RequestMapping("/goods")
public class GoodsController {
@Resource
private StoryEngine storyEngine;
@PostMapping("/show")
public GoodsDetail showGoods(@RequestBody GoodsDetailRequest request) {
StoryRequest<GoodsDetail> req = ReqBuilder.returnType(GoodsDetail.class).startId("kstry-demo-goods-show")
.trackingType(TrackingTypeEnum.SERVICE_DETAIL).request(request).build();
req.setRecallStoryHook(recallStory -> {
MonitorTracking monitorTracking = recallStory.getMonitorTracking();
List<NodeTracking> storyTracking = monitorTracking.getStoryTracking();
List<String> collect = storyTracking.stream()
.map(nt -> GlobalUtil.format("{}({})", nt.getNodeName(), nt.getSpendTime())).collect(Collectors.toList());
System.out.println("name list: " + String.join(",", collect));
});
TaskResponse<GoodsDetail> fire = storyEngine.fire(req);
if (fire.isSuccess()) {
return fire.getResult();
}
return null;
}
}
// 日志： name list: 初始化\n基本信息(73),风控服务(33),三方服务统计(1),加载SKU信息(8),加载广告(null),送运费险(30),加载收藏数(6),加载下单数(6),加载评价数(2),加载店铺信息(19),商详后置处理(15)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29