onnx转engine：
zjx@nvidia:/media/zjx/76524d80-8abc-4b9c-a262-a82f0f7c0b43/cuda/test$ /usr/src/tensorrt/bin/trtexec --onnx=retinanet-9.onnx --saveEngine=retinanet-9-2.engine &&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=retinanet-9.onnx --saveEngine=retinanet-9-2.engine [10/21/2020-19:35:53] [I] === Model Options === [10/21/2020-19:35:53] [I] Format: ONNX [10/21/2020-19:35:53] [I] Model: retinanet-9.onnx [10/21/2020-19:35:53] [I] Output: [10/21/2020-19:35:53] [I] === Build Options === [10/21/2020-19:35:53] [I] Max batch: 1 [10/21/2020-19:35:53] [I] Workspace: 16 MB [10/21/2020-19:35:53] [I] minTiming: 1 [10/21/2020-19:35:53] [I] avgTiming: 8 [10/21/2020-19:35:53] [I] Precision: FP32 [10/21/2020-19:35:53] [I] Calibration: [10/21/2020-19:35:53] [I] Safe mode: Disabled [10/21/2020-19:35:53] [I] Save engine: retinanet-9-2.engine [10/21/2020-19:35:53] [I] Load engine: [10/21/2020-19:35:53] [I] Builder Cache: Enabled [10/21/2020-19:35:53] [I] NVTX verbosity: 0 [10/21/2020-19:35:53] [I] Inputs format: fp32:CHW [10/21/2020-19:35:53] [I] Outputs format: fp32:CHW [10/21/2020-19:35:53] [I] Input build shapes: model [10/21/2020-19:35:53] [I] Input calibration shapes: model [10/21/2020-19:35:53] [I] === System Options === [10/21/2020-19:35:53] [I] Device: 0 [10/21/2020-19:35:53] [I] DLACore: [10/21/2020-19:35:53] [I] Plugins: [10/21/2020-19:35:53] [I] === Inference Options === [10/21/2020-19:35:53] [I] Batch: 1 [10/21/2020-19:35:53] [I] Input inference shapes: model [10/21/2020-19:35:53] [I] Iterations: 10 [10/21/2020-19:35:53] [I] Duration: 3s (+ 200ms warm up) [10/21/2020-19:35:53] [I] Sleep time: 0ms [10/21/2020-19:35:53] [I] Streams: 1 [10/21/2020-19:35:53] [I] ExposeDMA: Disabled [10/21/2020-19:35:53] [I] Spin-wait: Disabled [10/21/2020-19:35:53] [I] Multithreading: Disabled [10/21/2020-19:35:53] [I] CUDA Graph: Disabled [10/21/2020-19:35:53] [I] Skip inference: Disabled [10/21/2020-19:35:53] [I] Inputs: [10/21/2020-19:35:53] [I] === Reporting Options === [10/21/2020-19:35:53] [I] Verbose: Disabled [10/21/2020-19:35:53] [I] Averages: 10 inferences [10/21/2020-19:35:53] [I] Percentile: 99 [10/21/2020-19:35:53] [I] Dump output: Disabled [10/21/2020-19:35:53] [I] Profile: Disabled [10/21/2020-19:35:53] [I] Export timing to JSON file: [10/21/2020-19:35:53] [I] Export output to JSON file: [10/21/2020-19:35:53] [I] Export profile to JSON file: [10/21/2020-19:35:53] [I] ---------------------------------------------------------------- Input filename: retinanet-9.onnx ONNX IR version: 0.0.6 Opset version: 9 Producer name: pytorch Producer version: 1.6 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- [10/21/2020-19:36:04] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [10/21/2020-19:38:44] [I] [TRT] Detected 1 inputs and 10 output network tensors. [10/21/2020-19:38:53] [I] Starting inference threads [10/21/2020-19:38:57] [I] Warmup completed 1 queries over 200 ms [10/21/2020-19:38:57] [I] Timing trace has 11 queries over 3.79041 s [10/21/2020-19:38:57] [I] Trace averages of 10 runs: [10/21/2020-19:38:57] [I] Average on 10 runs - GPU latency: 343.479 ms - Host latency: 344.777 ms (end to end 344.786 ms, enqueue 11.3544 ms) [10/21/2020-19:38:57] [I] Host Latency [10/21/2020-19:38:57] [I] min: 342.54 ms (end to end 342.545 ms) [10/21/2020-19:38:57] [I] max: 346.81 ms (end to end 346.819 ms) [10/21/2020-19:38:57] [I] mean: 344.574 ms (end to end 344.582 ms) [10/21/2020-19:38:57] [I] median: 344.348 ms (end to end 344.355 ms) [10/21/2020-19:38:57] [I] percentile: 346.81 ms at 99% (end to end 346.819 ms at 99%) [10/21/2020-19:38:57] [I] throughput: 2.90206 qps [10/21/2020-19:38:57] [I] walltime: 3.79041 s [10/21/2020-19:38:57] [I] Enqueue Time [10/21/2020-19:38:57] [I] min: 3.94407 ms [10/21/2020-19:38:57] [I] max: 12.996 ms [10/21/2020-19:38:57] [I] median: 12.1548 ms [10/21/2020-19:38:57] [I] GPU Compute [10/21/2020-19:38:57] [I] min: 341.245 ms [10/21/2020-19:38:57] [I] max: 345.518 ms [10/21/2020-19:38:57] [I] mean: 343.275 ms [10/21/2020-19:38:57] [I] median: 343.049 ms [10/21/2020-19:38:57] [I] percentile: 345.518 ms at 99% [10/21/2020-19:38:57] [I] total compute time: 3.77603 s &&&& PASSED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=retinanet-9.onnx --saveEngine=retinanet-9-2.engine