网站顶部导航,东莞厂房招标平台,app定制开发收费,长春城乡建设部网站首页前言
我们在上一篇文章基于压测进行Feign调优完成的服务间调用的性能调优#xff0c;此时我们也关注到一个问题#xff0c;如果我们统一从网关调用服务#xff0c;但是网关因为某些原因报错或者没有找到服务怎么办呢#xff1f;
如下所示#xff0c;笔者通过网关调用acc…前言
我们在上一篇文章基于压测进行Feign调优完成的服务间调用的性能调优此时我们也关注到一个问题如果我们统一从网关调用服务但是网关因为某些原因报错或者没有找到服务怎么办呢
如下所示笔者通过网关调用account服务但是account服务还没起来。此时请求还没有到达account就报错了这就意味着我们服务中编写的RestControllerAdvice对网关没有任何作用。
curl 127.0.0.1:8090/account/getByCode/zsy
响应结果如下可以看到响应结果如下所示要知道现如今的开发模式为前后端分离模式前后端交互完全是基于协商好的格式如果网关响应格式与我们规定的格式完全不一致前端就需要特殊处理这使得代码不仅会变得丑陋对于后续的功能扩展的交互复杂度也会增加,而gateway默认响应错误如下:
{timestamp:2023-02-09T15:22:20.2780000,path:/account/getByCode/zsy,status:500,error:Internal Server Error,message:Connection refused: no further information: /192.168.43.73:9000
}网关异常默认处理
所以我们必须了解一下是什么原因导致网关报错会响应这个值。
我们在gateway源码中找到ErrorWebFluxAutoConfiguration这个自动装配类可以看到下面这段代码我们从中得知网关报错时默认使用DefaultErrorWebExceptionHandler 来返回结果所以我们不妨看看这个类做了那些事情。
BeanConditionalOnMissingBean(value ErrorWebExceptionHandler.class, search SearchStrategy.CURRENT)Order(-1)public ErrorWebExceptionHandler errorWebExceptionHandler(ErrorAttributes errorAttributes) {//网关默认异常处理的handlerDefaultErrorWebExceptionHandler exceptionHandler new DefaultErrorWebExceptionHandler(errorAttributes,this.resourceProperties, this.serverProperties.getError(), this.applicationContext);exceptionHandler.setViewResolvers(this.viewResolvers);exceptionHandler.setMessageWriters(this.serverCodecConfigurer.getWriters());exceptionHandler.setMessageReaders(this.serverCodecConfigurer.getReaders());return exceptionHandler;}我们不妨基于debug了解一下这个类当我们服务没有注册到nacos并通过网关调用报错时代码就会走到下方route 方法第一个参数是RequestPredicate谓词而后者则是谓词的处理进行renderErrorViewandRoute同理将报错的请求通过renderErrorResponse返回错误结果
Override
//route 方法第一个参数是RequestPredicate谓词而后者则是谓词的处理进行renderErrorView然后通过然后通过andRoute将报错的请求通过renderErrorResponse返回错误结果protected RouterFunctionServerResponse getRoutingFunction(ErrorAttributes errorAttributes) {return route(acceptsTextHtml(), this::renderErrorView).andRoute(all(), this::renderErrorResponse);}我们不妨看看renderErrorResponse可以看到一行getErrorAttributes一旦步入我们就可以看到上文请求错误的结果格式
protected MonoServerResponse renderErrorResponse(ServerRequest request) {boolean includeStackTrace isIncludeStackTrace(request, MediaType.ALL);MapString, Object error getErrorAttributes(request, includeStackTrace);return ServerResponse.status(getHttpStatus(error)).contentType(MediaType.APPLICATION_JSON_UTF8).body(BodyInserters.fromObject(error));}getErrorAttributes源码可以看到组装的key值就是我们调试时响应的参数
Overridepublic MapString, Object getErrorAttributes(ServerRequest request, boolean includeStackTrace) {MapString, Object errorAttributes new LinkedHashMap();errorAttributes.put(timestamp, new Date());errorAttributes.put(path, request.path());Throwable error getError(request);HttpStatus errorStatus determineHttpStatus(error);errorAttributes.put(status, errorStatus.value());errorAttributes.put(error, errorStatus.getReasonPhrase());errorAttributes.put(message, determineMessage(error));handleException(errorAttributes, determineException(error), includeStackTrace);return errorAttributes;}自定义异常处理
了解的默认错误处理我们就可以改造返回一个和普通服务一样的格式给前端告知网关报错。从上文我们可知网关默认错误处理时DefaultErrorWebExceptionHandler通过类图我们可以发现它继承了一个ErrorWebExceptionHandler所以我们也可以继承这个类重写一个Handler。 以笔者的代码如下可以看到笔者使用Order注解强制获得最高异常处理优先级然后使用bufferFactory.wrap方法传递自定义错误格式返回给前端。
Slf4j
Order(-1)
Configuration
RequiredArgsConstructor(onConstructor __(Autowired))
public class GlobalErrorWebExceptionHandler implements ErrorWebExceptionHandler {private final ObjectMapper objectMapper;Overridepublic MonoVoid handle(ServerWebExchange exchange, Throwable ex) {ServerHttpResponse response exchange.getResponse();if (response.isCommitted()) {return Mono.error(ex);}// 设置返回值类型为jsonresponse.getHeaders().setContentType(MediaType.APPLICATION_JSON);//设置返回编码if (ex instanceof ResponseStatusException) {response.setStatusCode(((ResponseStatusException) ex).getStatus());}return response.writeWith(Mono.fromSupplier(() - {DataBufferFactory bufferFactory response.bufferFactory();try {//writeValueAsBytes 组装错误响应结果return bufferFactory.wrap(objectMapper.writeValueAsBytes(ResultData.fail(500, 网关捕获到异常: ex.getMessage())));} catch (JsonProcessingException e) {log.error(Error writing response, ex);return bufferFactory.wrap(new byte[0]);}}));}
}最终返回的结果如下所示可以看到结果和一般的服务调用报错格式一模一样,这样一来前端就无需为了网关报错加一个特殊处理的逻辑了
curl 127.0.0.1:8090/account/getByCode/zsy
输出结果
{status:500,message:网关捕获到异常:503 SERVICE_UNAVAILABLE \Unable to find instance for account-service\,data:null,success:false,timestamp:1675959617386
}请求响应日志监控
对于微服务架构来说监控是很重要的在高并发场景情况下很多问题我们都可以在网关请求响应中定位到所以我们希望能有这么一种方式将用户日常请求响应的日志信息记录下来便于日常运维和性能监控。
查阅了网上的资料发现基于MongoDB进行网关请求响应数据采集是一种不错的方案所以笔者本篇文章整理一下笔者如何基于网关过滤器结合MongoDB完成请求日志采集。
本篇文章可能会涉及MongoDB相关的知识不了解的读者可以参考笔者的这篇文章:
MongoDB快速入门
gateway整合MongoDB采集日志步骤
添加MongoDB依赖并完成MongoDB配置:
首先在gateway中添加MongoDB依赖需要注意的是笔者后续的过滤器某些代码段会用到hutool的工具类所以这里也添加了hutool的依赖。 !--mongodb依赖--dependencygroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-data-mongodb-reactive/artifactIdexclusionsexclusiongroupIdorg.springframework.boot/groupIdartifactIdspring-boot-starter-logging/artifactId/exclusion/exclusions/dependencydependencygroupIdcn.hutool/groupIdartifactIdhutool-all/artifactId/dependency然后我们在gateway的配置中添加MongoDB的连接参数配置:
# mongodb的ip地址
spring.data.mongodb.hostip
# mongodb端口号
spring.data.mongodb.port27017
# mongodb数据库名称
spring.data.mongodb.databaseaccesslog
# 用户名
spring.data.mongodb.usernamexxxx
# 密码
spring.data.mongodb.passwordxxx编写MongoDB保存逻辑:
我们希望保存网关响应的内容到mongodb中所以我们要把我们需要的内容封装成一个对象如下GatewayLog
Data
public class GatewayLog {/*** 请求相对路径*/private String requestPath;/***请求方法 :get post*/private String requestMethod;/***请求协议:http rpc*/private String schema;/***请求体内容*/private String requestBody;/***响应内容*/private String responseBody;/***ip地址*/private String ip;/*** 请求时间*/private String requestTime;/***响应时间*/private String responseTime;/***执行时间 单位:毫秒*/private Long executeTime;}
完成对象定义后我们就可以编写service层接口和实现类的逻辑了:
public interface AccessLogService {/*** 保存AccessLog* param gatewayLog 请求响应日志* return 响应日志*/GatewayLog saveAccessLog(GatewayLog gatewayLog);}实现类代码如下可以看到笔者完全基于mongoTemplate的save方法将日志数据存到gatewayLog表中。
Service
public class AccessLogServiceImpl implements AccessLogService {Autowiredprivate MongoTemplate mongoTemplate;//collection名称private final String collectionNamegatewayLog ;Overridepublic GatewayLog saveAccessLog(GatewayLog gatewayLog) {GatewayLog result mongoTemplate.save(gatewayLog, collectionName);return result;}
}基于gateway过滤器完成请求相应日志采集,代码比较长首先是CachedBodyOutputMessage由于笔者用的是Spring boot 2.x版本没有CachedBodyOutputMessage 这个类所以笔者从网上找了一份。读者可以根据注释进行复制修改即可。
public class CachedBodyOutputMessage implements ReactiveHttpOutputMessage {private final DataBufferFactory bufferFactory;private final HttpHeaders httpHeaders;private FluxDataBuffer body Flux.error(new IllegalStateException(The body is not set. Did handling complete with success? Is a custom \writeHandler\ configured?));private FunctionFluxDataBuffer, MonoVoid writeHandler this.initDefaultWriteHandler();public CachedBodyOutputMessage(ServerWebExchange exchange, HttpHeaders httpHeaders) {this.bufferFactory exchange.getResponse().bufferFactory();this.httpHeaders httpHeaders;}public void beforeCommit(Supplier? extends MonoVoid action) {}public boolean isCommitted() {return false;}public HttpHeaders getHeaders() {return this.httpHeaders;}private FunctionFluxDataBuffer, MonoVoid initDefaultWriteHandler() {return (body) - {this.body body.cache();return this.body.then();};}public DataBufferFactory bufferFactory() {return this.bufferFactory;}public FluxDataBuffer getBody() {return this.body;}public void setWriteHandler(FunctionFluxDataBuffer, MonoVoid writeHandler) {Assert.notNull(writeHandler, writeHandler is required);this.writeHandler writeHandler;}public MonoVoid writeWith(Publisher? extends DataBuffer body) {return Mono.defer(() - {return (Mono)this.writeHandler.apply(Flux.from(body));});}public MonoVoid writeAndFlushWith(Publisher? extends Publisher? extends DataBuffer body) {return this.writeWith(Flux.from(body).flatMap((p) - {return p;}));}public MonoVoid setComplete() {return this.writeWith(Flux.empty());}
}过滤器代码如下笔者将核心内容都已注释了读者可以基于此代码进行修改 Slf4j
Component
public class AccessLogGlobalFilter implements GlobalFilter, Ordered {private final ListHttpMessageReader? messageReaders HandlerStrategies.withDefaults().messageReaders();//todo 存在线程安全问题,后续需要优化掉SimpleDateFormat simpleDateFormat new SimpleDateFormat(yyyy-MM-dd HH:mm:ss.SSS);Autowiredprivate AccessLogService accessLogService;Overridepublic MonoVoid filter(ServerWebExchange exchange, GatewayFilterChain chain) {GatewayLog gatewayLog new GatewayLog();ServerHttpRequest request exchange.getRequest();//获取请求的ipurlmethodbodyString requestPath request.getPath().pathWithinApplication().value();String clientIp request.getRemoteAddress().getHostString();String scheme request.getURI().getScheme();String method request.getMethodValue();//数据记录到gatwayLog中gatewayLog.setSchema(scheme);gatewayLog.setRequestMethod(method);gatewayLog.setRequestPath(requestPath);gatewayLog.setRequestTime(simpleDateFormat.format(new Date().getTime()));gatewayLog.setIp(clientIp);MediaType contentType request.getHeaders().getContentType();if (MediaType.APPLICATION_FORM_URLENCODED.isCompatibleWith(contentType) || MediaType.APPLICATION_JSON.isCompatibleWith(contentType)) {return writeBodyLog(exchange, chain, gatewayLog);} else {//写入日志信息到mongoDbreturn writeBasicLog(exchange, chain, gatewayLog);}}private MonoVoid writeBasicLog(ServerWebExchange exchange, GatewayFilterChain chain, GatewayLog accessLog) {StringBuilder builder new StringBuilder();MultiValueMapString, String queryParams exchange.getRequest().getQueryParams();for (Map.EntryString, ListString entry : queryParams.entrySet()) {builder.append(entry.getKey()).append().append(StringUtils.join(entry.getValue(), ,));}//记录响应内容accessLog.setRequestBody(builder.toString());// 获取响应体ServerHttpResponseDecorator decoratedResponse recordResponseLog(exchange, accessLog);return chain.filter(exchange.mutate().response(decoratedResponse).build()).then(Mono.fromRunnable(() - {//打印日志writeAccessLog(accessLog);}));}/*** 解决request body 只能读取一次问题** param exchange* param chain* param gatewayLog* return*/private Mono writeBodyLog(ServerWebExchange exchange, GatewayFilterChain chain, GatewayLog gatewayLog) {ServerRequest serverRequest ServerRequest.create(exchange, messageReaders);MonoString modifiedBody serverRequest.bodyToMono(String.class).flatMap(body - {gatewayLog.setRequestBody(body);return Mono.just(body);});// 通过 BodyInsert 插入 body(支持修改body), 避免 request body 只能获取一次BodyInserter bodyInserter BodyInserters.fromPublisher(modifiedBody, String.class);HttpHeaders headers new HttpHeaders();headers.putAll(exchange.getRequest().getHeaders());headers.remove(HttpHeaders.CONTENT_LENGTH);CachedBodyOutputMessage outputMessage new CachedBodyOutputMessage(exchange, headers);return bodyInserter.insert(outputMessage, new BodyInserterContext()).then(Mono.defer(() - {// 重新封装请求ServerHttpRequest decoratedRequest requestDecorate(exchange, headers, outputMessage);// 记录响应日志ServerHttpResponseDecorator decoratedResponse recordResponseLog(exchange, gatewayLog);// 记录普通的return chain.filter(exchange.mutate().request(decoratedRequest).response(decoratedResponse).build()).then(Mono.fromRunnable(() - {// 打印日志writeAccessLog(gatewayLog);}));}));}/*** 打印日志并将日志内容写入mongodb** param gatewayLog*/private void writeAccessLog(GatewayLog gatewayLog) {log.info(写入网关日志日志内容: JSON.toJSONString(gatewayLog));accessLogService.saveAccessLog(gatewayLog);}/*** 请求装饰器重新计算 headers** param exchange* param headers* param outputMessage* return*/private ServerHttpRequestDecorator requestDecorate(ServerWebExchange exchange, HttpHeaders headers,CachedBodyOutputMessage outputMessage) {return new ServerHttpRequestDecorator(exchange.getRequest()) {Overridepublic HttpHeaders getHeaders() {long contentLength headers.getContentLength();HttpHeaders httpHeaders new HttpHeaders();httpHeaders.putAll(super.getHeaders());if (contentLength 0) {httpHeaders.setContentLength(contentLength);} else {httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, chunked);}return httpHeaders;}Overridepublic FluxDataBuffer getBody() {return outputMessage.getBody();}};}/*** 记录响应日志** param exchange* param gatewayLog* return*/private ServerHttpResponseDecorator recordResponseLog(ServerWebExchange exchange, GatewayLog gatewayLog) {ServerHttpResponse response exchange.getResponse();DataBufferFactory bufferFactory response.bufferFactory();return new ServerHttpResponseDecorator(response) {SneakyThrowsOverridepublic MonoVoid writeWith(Publisher? extends DataBuffer body) {if (body instanceof Flux) {String responseTime simpleDateFormat.format(new Date().getTime());gatewayLog.setResponseTime(responseTime);// 计算执行时间long executeTime (simpleDateFormat.parse(responseTime).getTime() - simpleDateFormat.parse(gatewayLog.getRequestTime()).getTime());gatewayLog.setExecuteTime(executeTime);// 获取响应类型如果是 json 就打印String originalResponseContentType exchange.getAttribute(ServerWebExchangeUtils.ORIGINAL_RESPONSE_CONTENT_TYPE_ATTR);if (ObjectUtils.equals(this.getStatusCode(), HttpStatus.OK) StringUtils.isNotBlank(originalResponseContentType) originalResponseContentType.contains(application/json)) {Flux? extends DataBuffer fluxBody Flux.from(body);return super.writeWith(fluxBody.buffer().map(dataBuffers - {// 合并多个流集合解决返回体分段传输DataBufferFactory dataBufferFactory new DefaultDataBufferFactory();DataBuffer join dataBufferFactory.join(dataBuffers);byte[] content new byte[join.readableByteCount()];join.read(content);// 释放掉内存DataBufferUtils.release(join);String responseResult new String(content, StandardCharsets.UTF_8);gatewayLog.setResponseBody(responseResult);return bufferFactory.wrap(content);}));}}return super.writeWith(body);}};}/*** 调小优先级使得该过滤器最先执行* return*/Overridepublic int getOrder() {return -100;}
}
测试
以笔者项目为例通过网关调用order服务
curl 127.0.0.1:8090/order/getByCode/zsy可以看到响应成功了接下来我们就确认一下mongoDb中是否有存储网关请求响应信息 {status:100,message:操作成功,data:{id:1,accountCode:zsy,accountName:zsy,amount:10000.00},success:true,timestamp:1676439102837}
通过数据库连接工具查询可以看到网关请求响应日志也成功存储到MongoDB中。 参考文献
SpringCloud Alibaba微服务实战二十四 - SpringCloud Gateway的全局异常处理
软件开发设计中的上游与下游
SpringCloud Alibaba实战二十九 | SpringCloud Gateway 请求响应日志
MongoDB 数据查询操作
实战 | MongoDB的安装配置
spring cloud gateway中实现请求、响应参数日志打印
