Skip to content

Commit de72cb8

Browse files
BruceForstallBruceForstall
authored
Enable SuperPMI replay for APX ISA testing (dotnet#113843)
* Don't fail altjit compiles when RunAltJitCode=0 With RunAltJitCode=0, the JIT returns CORJIT_SKIPPED. We don't want superpmi to fail these compiles. * Enable SuperPMI replay for APX ISA testing Create a new `runtime-coreclr superpmi-replay-apx` pipeline that does SuperPMI replay an AltJit with various APX enabling configuration options set. This enables some amount of APX testing even without hardware available to run the generated code. Notes: 1. Create a new jit-replay-pipeline.yml template to share the superpmi-replay pipeline implementation with the existing pipeline. 2. The existing pipeline is designated the "standard" replay type and the new pipeline the "apx" replay type. 3. The APX pipeline does replays for Windows x64 and Linux x64. 4. The configurations tested are: ``` a. "RunAltJitCode=0", "EnableAPX=1", "EnableApxNDD=1" b. "RunAltJitCode=0", "EnableAPX=1", "JitStressRex2Encoding=1" c. "RunAltJitCode=0", "EnableAPX=1", "JitStressPromotedEvexEncoding=1" d. "RunAltJitCode=0", "EnableAPX=1", "JitStressRegs=4000" e. "RunAltJitCode=0", "EnableAPX=1", "EnableApxNDD=1", "JitStressRex2Encoding=1", "JitStressPromotedEvexEncoding=1", "JitStressRegs=4000" ```
1 parent 45a0e10 commit de72cb8

File tree

14 files changed

+234
-110
lines changed

14 files changed

+234
-110
lines changed

‎eng/pipelines/common/templates/runtimes/send-to-helix-step.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ parameters:
3030
runtimeVariant: ''
3131
shouldContinueOnError: false
3232
SuperPmiCollect: ''
33+
SuperPmiReplayType: ''
3334
SuperPmiDiffType: ''
3435
SuperPmiBaseJitOptions: ''
3536
SuperPmiDiffJitOptions: ''
@@ -65,6 +66,7 @@ steps:
6566
RuntimeFlavor: ${{ parameters.runtimeFlavor }}
6667
_RuntimeVariant: ${{ parameters.runtimeVariant }}
6768
_SuperPmiCollect: ${{ parameters.SuperPmiCollect }}
69+
_SuperPmiReplayType: ${{ parameters.SuperPmiReplayType }}
6870
_SuperPmiDiffType: ${{ parameters.SuperPmiDiffType }}
6971
_SuperPmiBaseJitOptions: ${{ parameters.SuperPmiBaseJitOptions }}
7072
_SuperPmiDiffJitOptions: ${{ parameters.SuperPmiDiffJitOptions }}

‎eng/pipelines/coreclr/superpmi-replay-apx.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
trigger: none
2+
3+
schedules:
4+
- cron: "0 7 * * *"
5+
displayName: Daily at 11:00 PM (UTC-8:00)
6+
branches:
7+
include:
8+
- main
9+
always: true
10+
11+
variables:
12+
- template: /eng/pipelines/common/variables.yml
13+
14+
extends:
15+
template: /eng/pipelines/coreclr/templates/jit-replay-pipeline.yml
16+
parameters:
17+
platforms:
18+
- windows_x64
19+
replayType: apx

‎eng/pipelines/coreclr/superpmi-replay.yml‎

Lines changed: 5 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -20,55 +20,9 @@ variables:
2020
- template: /eng/pipelines/common/variables.yml
2121

2222
extends:
23-
template: /eng/pipelines/common/templates/pipeline-with-resources.yml
23+
template: /eng/pipelines/coreclr/templates/jit-replay-pipeline.yml
2424
parameters:
25-
stages:
26-
# Don't run if the JIT-EE GUID has changed,
27-
# since there won't be any SuperPMI collections with the new GUID until the collection
28-
# pipeline completes after this PR is merged.
29-
- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
30-
- stage: EvaluatePaths
31-
displayName: Evaluate Paths
32-
jobs:
33-
- template: /eng/pipelines/common/evaluate-paths-job.yml
34-
parameters:
35-
paths:
36-
- subset: jiteeversionguid
37-
include:
38-
- src/coreclr/inc/jiteeversionguid.h
39-
40-
- stage: Build
41-
jobs:
42-
43-
- template: /eng/pipelines/common/platform-matrix.yml
44-
parameters:
45-
jobTemplate: /eng/pipelines/common/global-build-job.yml
46-
buildConfig: checked
47-
platforms:
48-
- windows_x64
49-
- windows_x86
50-
jobParameters:
51-
buildArgs: -s clr.alljits+clr.spmi -c $(_BuildConfig)
52-
postBuildSteps:
53-
- template: /eng/pipelines/common/upload-artifact-step.yml
54-
parameters:
55-
rootFolder: $(Build.SourcesDirectory)/artifacts/bin/coreclr
56-
includeRootFolder: false
57-
archiveType: $(archiveType)
58-
tarCompression: $(tarCompression)
59-
archiveExtension: $(archiveExtension)
60-
artifactName: CheckedJIT_$(osGroup)$(osSubgroup)_$(archType)
61-
displayName: JIT and SuperPMI Assets
62-
condition: not(eq(stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_jiteeversionguid.containsChange'], true))
63-
64-
- template: /eng/pipelines/common/platform-matrix.yml
65-
parameters:
66-
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-replay-job.yml
67-
buildConfig: checked
68-
platforms:
69-
- windows_x64
70-
- windows_x86
71-
helixQueueGroup: ci
72-
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
73-
jobParameters:
74-
condition: not(eq(stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_jiteeversionguid.containsChange'], true))
25+
platforms:
26+
- windows_x64
27+
- windows_x86
28+
replayType: standard

‎eng/pipelines/coreclr/templates/jit-replay-pipeline.yml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
parameters:
2+
- name: platforms
3+
type: object
4+
- name: replayType
5+
type: string
6+
default: standard
7+
8+
extends:
9+
template: /eng/pipelines/common/templates/pipeline-with-resources.yml
10+
parameters:
11+
stages:
12+
# Don't run if the JIT-EE GUID has changed,
13+
# since there won't be any SuperPMI collections with the new GUID until the collection
14+
# pipeline completes after this PR is merged.
15+
- ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
16+
- stage: EvaluatePaths
17+
displayName: Evaluate Paths
18+
jobs:
19+
- template: /eng/pipelines/common/evaluate-paths-job.yml
20+
parameters:
21+
paths:
22+
- subset: jiteeversionguid
23+
include:
24+
- src/coreclr/inc/jiteeversionguid.h
25+
26+
- stage: Build
27+
jobs:
28+
29+
- template: /eng/pipelines/common/platform-matrix.yml
30+
parameters:
31+
jobTemplate: /eng/pipelines/common/global-build-job.yml
32+
buildConfig: checked
33+
platforms: ${{ parameters.platforms }}
34+
jobParameters:
35+
buildArgs: -s clr.alljits+clr.spmi -c $(_BuildConfig)
36+
postBuildSteps:
37+
- template: /eng/pipelines/common/upload-artifact-step.yml
38+
parameters:
39+
rootFolder: $(Build.SourcesDirectory)/artifacts/bin/coreclr
40+
includeRootFolder: false
41+
archiveType: $(archiveType)
42+
tarCompression: $(tarCompression)
43+
archiveExtension: $(archiveExtension)
44+
artifactName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)
45+
displayName: Build Assets
46+
condition: not(eq(stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_jiteeversionguid.containsChange'], true))
47+
48+
- template: /eng/pipelines/common/platform-matrix.yml
49+
parameters:
50+
jobTemplate: /eng/pipelines/coreclr/templates/superpmi-replay-job.yml
51+
buildConfig: checked
52+
platforms: ${{ parameters.platforms }}
53+
helixQueueGroup: ci
54+
helixQueuesTemplate: /eng/pipelines/coreclr/templates/helix-queues-setup.yml
55+
jobParameters:
56+
condition: not(eq(stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_jiteeversionguid.containsChange'], true))
57+
replayType: ${{ parameters.replayType }}
58+
unifiedArtifactsName: BuildArtifacts_$(osGroup)$(osSubgroup)_$(archType)_$(_BuildConfig)

‎eng/pipelines/coreclr/templates/run-superpmi-replay-job.yml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ parameters:
1616
enableTelemetry: false # optional -- enable for telemetry
1717
liveLibrariesBuildConfig: '' # optional -- live-live libraries configuration to use for the run
1818
helixQueues: '' # required -- Helix queues
19+
replayType: 'standard' # required -- 'standard', 'apx'
1920

2021
jobs:
2122
- template: /eng/pipelines/common/templates/runtimes/xplat-job.yml
@@ -47,6 +48,9 @@ jobs:
4748
- ${{ each variable in parameters.variables }}:
4849
- ${{insert}}: ${{ variable }}
4950

51+
- name: replayType
52+
value: ${{ parameters.replayType }}
53+
5054
- template: /eng/pipelines/coreclr/templates/jit-python-variables.yml
5155
parameters:
5256
osGroup: ${{ parameters.osGroup }}
@@ -74,8 +78,8 @@ jobs:
7478
mkdir $(SpmiLogsLocation)
7579
displayName: Create directories
7680
77-
- script: $(PythonScript) $(Build.SourcesDirectory)/src/coreclr/scripts/superpmi_replay_setup.py -source_directory $(Build.SourcesDirectory) -product_directory $(buildProductRootFolderPath) -arch $(archType)
78-
displayName: ${{ format('SuperPMI replay setup ({0})', parameters.archType) }}
81+
- script: $(PythonScript) $(Build.SourcesDirectory)/src/coreclr/scripts/superpmi_replay_setup.py -source_directory $(Build.SourcesDirectory) -product_directory $(buildProductRootFolderPath) -type $(replayType) -arch $(archType)
82+
displayName: ${{ format('SuperPMI replay setup ({0} {1})', parameters.replayType, parameters.archType) }}
7983

8084
# Run superpmi replay in helix
8185
- template: /eng/pipelines/common/templates/runtimes/send-to-helix-step.yml
@@ -93,6 +97,7 @@ jobs:
9397
BuildConfig: ${{ parameters.buildConfig }}
9498
osGroup: ${{ parameters.osGroup }}
9599
archType: ${{ parameters.archType }}
100+
SuperPmiReplayType: ${{ parameters.replayType }}
96101

97102
# Always upload the available logs for diagnostics
98103
- task: CopyFiles@2

‎eng/pipelines/coreclr/templates/superpmi-replay-job.yml‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@ parameters:
99
variables: {}
1010
helixQueues: ''
1111
runJobTemplate: '/eng/pipelines/coreclr/templates/run-superpmi-replay-job.yml'
12+
replayType: 'standard'
13+
unifiedArtifactsName: ''
1214

1315
jobs:
1416
- template: ${{ parameters.runJobTemplate }}
1517
parameters:
16-
jobName: ${{ format('superpmi_replay_{0}{1}_{2}_{3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
17-
displayName: ${{ format('SuperPMI replay {0}{1} {2} {3}', parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
18+
jobName: ${{ format('superpmi_replay_{0}_{1}{2}_{3}_{4}', parameters.replayType, parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
19+
displayName: ${{ format('SuperPMI replay {0} {1}{2} {3} {4}', parameters.replayType, parameters.osGroup, parameters.osSubgroup, parameters.archType, parameters.buildConfig) }}
1820
pool: ${{ parameters.pool }}
1921
buildConfig: ${{ parameters.buildConfig }}
2022
archType: ${{ parameters.archType }}
@@ -23,18 +25,19 @@ jobs:
2325
condition: ${{ parameters.condition }}
2426
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
2527
helixQueues: ${{ parameters.helixQueues }}
28+
replayType: ${{ parameters.replayType }}
2629
dependsOn:
2730
- 'build_${{ parameters.osGroup }}${{ parameters.osSubgroup }}_${{ parameters.archType }}_${{ parameters.buildConfig }}_'
2831

2932
variables: ${{ parameters.variables }}
3033

3134
steps:
3235

33-
# Download jit builds
36+
# Download builds
3437
- template: /eng/pipelines/common/download-artifact-step.yml
3538
parameters:
3639
unpackFolder: $(Build.SourcesDirectory)/artifacts/bin/coreclr
37-
artifactFileName: 'CheckedJIT_$(osGroup)$(osSubgroup)_$(archType)$(archiveExtension)'
38-
artifactName: 'CheckedJIT_$(osGroup)$(osSubgroup)_$(archType)'
39-
displayName: 'JIT checked build'
40+
artifactFileName: '${{ parameters.unifiedArtifactsName }}$(archiveExtension)'
41+
artifactName: '${{ parameters.unifiedArtifactsName }}'
42+
displayName: 'unified artifacts'
4043
cleanUnpackFolder: false

‎src/coreclr/scripts/superpmi-replay.proj‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,11 +24,17 @@
2424
timeout: %(HelixWorkItem.Timeout) '"/>
2525
</Target> -->
2626

27+
<PropertyGroup>
28+
<!-- Default to standard type -->
29+
<SuperPmiReplayType Condition=" '$(_SuperPmiReplayType)' == '' ">standard</SuperPmiReplayType>
30+
<SuperPmiReplayType Condition=" '$(_SuperPmiReplayType)' != '' ">$(_SuperPmiReplayType)</SuperPmiReplayType>
31+
</PropertyGroup>
32+
2733
<PropertyGroup>
2834
<Python>%HELIX_PYTHONPATH%</Python>
2935
<ProductDirectory>%HELIX_CORRELATION_PAYLOAD%</ProductDirectory>
3036
<SuperpmiLogsLocation>%HELIX_WORKITEM_UPLOAD_ROOT%</SuperpmiLogsLocation>
31-
<WorkItemCommand>$(Python) $(ProductDirectory)\superpmi_replay.py -jit_directory $(ProductDirectory)</WorkItemCommand>
37+
<WorkItemCommand>$(Python) $(ProductDirectory)\superpmi_replay.py -type $(SuperPmiReplayType) -jit_directory $(ProductDirectory)</WorkItemCommand>
3238
<WorkItemTimeout>3:15</WorkItemTimeout>
3339
</PropertyGroup>
3440

@@ -49,7 +55,7 @@
4955
</HelixCorrelationPayload>
5056
</ItemGroup>
5157

52-
<ItemGroup Condition="'$(Architecture)' == 'x64'">
58+
<ItemGroup Condition="'$(SuperPmiReplayType)' == 'standard' and '$(Architecture)' == 'x64'">
5359
<SPMI_Partition Include="windows-x64-1" Platform="windows" Architecture="x64" Partition="1" PartitionCount="9"/>
5460
<SPMI_Partition Include="windows-x64-2" Platform="windows" Architecture="x64" Partition="2" PartitionCount="9"/>
5561
<SPMI_Partition Include="windows-x64-3" Platform="windows" Architecture="x64" Partition="3" PartitionCount="9"/>
@@ -97,7 +103,7 @@
97103
<SPMI_Partition Include="osx-arm64-9" Platform="osx" Architecture="arm64" Partition="9" PartitionCount="9"/>
98104
</ItemGroup>
99105

100-
<ItemGroup Condition="'$(Architecture)' == 'x86'">
106+
<ItemGroup Condition="'$(SuperPmiReplayType)' == 'standard' and '$(Architecture)' == 'x86'">
101107
<SPMI_Partition Include="windows-x86-1" Platform="windows" Architecture="x86" Partition="1" PartitionCount="9"/>
102108
<SPMI_Partition Include="windows-x86-2" Platform="windows" Architecture="x86" Partition="2" PartitionCount="9"/>
103109
<SPMI_Partition Include="windows-x86-3" Platform="windows" Architecture="x86" Partition="3" PartitionCount="9"/>
@@ -118,11 +124,26 @@
118124
<SPMI_Partition Include="linux-arm-9" Platform="linux" Architecture="arm" Partition="9" PartitionCount="9"/>
119125
</ItemGroup>
120126

127+
<ItemGroup Condition="'$(SuperPmiReplayType)' == 'apx' and '$(Architecture)' == 'x64'">
128+
<SPMI_Partition Include="windows-x64-1" Platform="windows" Architecture="x64" Partition="1" PartitionCount="6"/>
129+
<SPMI_Partition Include="windows-x64-2" Platform="windows" Architecture="x64" Partition="2" PartitionCount="6"/>
130+
<SPMI_Partition Include="windows-x64-3" Platform="windows" Architecture="x64" Partition="3" PartitionCount="6"/>
131+
<SPMI_Partition Include="windows-x64-4" Platform="windows" Architecture="x64" Partition="4" PartitionCount="6"/>
132+
<SPMI_Partition Include="windows-x64-5" Platform="windows" Architecture="x64" Partition="5" PartitionCount="6"/>
133+
<SPMI_Partition Include="windows-x64-6" Platform="windows" Architecture="x64" Partition="6" PartitionCount="6"/>
134+
<SPMI_Partition Include="linux-x64-1" Platform="linux" Architecture="x64" Partition="1" PartitionCount="6"/>
135+
<SPMI_Partition Include="linux-x64-2" Platform="linux" Architecture="x64" Partition="2" PartitionCount="6"/>
136+
<SPMI_Partition Include="linux-x64-3" Platform="linux" Architecture="x64" Partition="3" PartitionCount="6"/>
137+
<SPMI_Partition Include="linux-x64-4" Platform="linux" Architecture="x64" Partition="4" PartitionCount="6"/>
138+
<SPMI_Partition Include="linux-x64-5" Platform="linux" Architecture="x64" Partition="5" PartitionCount="6"/>
139+
<SPMI_Partition Include="linux-x64-6" Platform="linux" Architecture="x64" Partition="6" PartitionCount="6"/>
140+
</ItemGroup>
141+
121142
<ItemGroup>
122143
<HelixWorkItem Include="@(SPMI_Partition)">
123144
<Command>$(WorkItemCommand) -arch %(HelixWorkItem.Architecture) -platform %(HelixWorkItem.Platform) -partition %(HelixWorkItem.Partition) -partition_count %(HelixWorkItem.PartitionCount) -log_directory $(SuperpmiLogsLocation)</Command>
124145
<Timeout>$(WorkItemTimeout)</Timeout>
125-
<DownloadFilesFromResults>superpmi_%(HelixWorkItem.Platform)_%(HelixWorkItem.Architecture)_%(HelixWorkItem.Partition).log</DownloadFilesFromResults>
146+
<DownloadFilesFromResults>superpmi_final_%(HelixWorkItem.Platform)_%(HelixWorkItem.Architecture)_%(HelixWorkItem.Partition).log</DownloadFilesFromResults>
126147
</HelixWorkItem>
127148
</ItemGroup>
128149
</Project>

0 commit comments

Comments
 (0)