infernap12/jdk
1
0
Fork 0
mirror of https://github.com/openjdk/jdk.git synced 2026-08-03 06:35:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'openjdk:master' into dwhite-simpler-apx-regression

Browse Source
This commit is contained in:
Derek White 2026-07-31 15:15:03 -04:00 committed by GitHub
commit 5e4014296a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2660 changed files with 234910 additions and 23005 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
doc/testing.html
View File

@ -465,6 +465,44 @@ class, named Virtual, is currently part of the JDK build in the
<code>test/jtreg_test_thread_factory/</code> directory. This class gets
compiled during the test image build. The implementation of the Virtual
class creates a new virtual thread for executing each test class.</p>
<h4 id="value_class_plugin">VALUE_CLASS_PLUGIN</h4>
<p>Enables the <code>ValueClassPlugin</code> javac plugin when compiling
and running JTReg tests. This is a <strong>temporary mode</strong>
intended for use while value classes (JEP 401) are a preview feature.
The long-term plan is to replace classes annotated with
<code>@jdk.test.lib.valueclass.AsValueClass</code> with plain
<code>value class</code> declarations once value classes are
finalized.</p>
<p>In the meantime, this mode allows test sources to compile and run as
either value classes or regular identity classes without source-level
changes.</p>
<p>When set to any non-empty value, the following options are appended to
every JTReg invocation:</p>
<ul>
<li><code>-cpa:&lt;valueClassPlugin.jar&gt;</code> — appends the plugin
JAR to the compile-time classpath (only when the JAR is present in the
test image under
<code>jtreg_value_class_plugin/valueClassPlugin.jar</code>).</li>
<li><code>-vmoption:--enable-preview</code> — enables JVM preview
features at runtime.</li>
<li><code>-javacoption:-XDaccessInternalAPI</code> — grants the compiler
access to internal APIs required by the plugin.</li>
<li><code>-javacoption:--source &lt;version&gt; --enable-preview</code>
— enables preview language features at compile time.</li>
<li><code>-javacoption:-Xplugin:ValueClassPlugin</code> — activates the
plugin.</li>
</ul>
<p>The plugin scans each compilation unit after parsing and converts any
class annotated with
<code>@jdk.test.lib.valueclass.AsValueClass</code> into a value class by
setting the internal <code>VALUE_CLASS</code> modifier flag and clearing
the <code>IDENTITY_TYPE</code> flag. This transformation only takes
effect when <code>--enable-preview</code> is active; without it the
annotation is a no-op and the class compiles as an ordinary identity
class, so the same test source can exercise both code paths.</p>
<p>Example:</p>
<pre><code>$ make test TEST=jdk_lang JTREG="VALUE_CLASS_PLUGIN=true"
</code></pre>
<h4 id="jvmti_stress_agent">JVMTI_STRESS_AGENT</h4>
<p>Executes JTReg tests with JVM TI stress agent. The stress agent is
the part of test library and located in

36
doc/testing.md
View File

@ -403,6 +403,42 @@ the `test/jtreg_test_thread_factory/` directory. This class gets compiled
during the test image build. The implementation of the Virtual class creates a
new virtual thread for executing each test class.
#### VALUE_CLASS_PLUGIN
Enables the `ValueClassPlugin` javac plugin when compiling and running JTReg
tests. This is a **temporary mode** intended for use while value classes
(JEP 401) are a preview feature. The long-term plan is to replace classes
annotated with `@jdk.test.lib.valueclass.AsValueClass` with plain
`value class` declarations once value classes are finalized.
In the meantime, this mode allows test sources to compile and run as either
value classes or regular identity classes without source-level changes.
When set to any non-empty value, the following options are appended to every
JTReg invocation:
* `-cpa:<valueClassPlugin.jar>` — appends the plugin JAR to the compile-time
classpath (only when the JAR is present in the test image under
`jtreg_value_class_plugin/valueClassPlugin.jar`).
* `-vmoption:--enable-preview` — enables JVM preview features at runtime.
* `-javacoption:-XDaccessInternalAPI` — grants the compiler access to internal
APIs required by the plugin.
* `-javacoption:--source <version> --enable-preview` — enables preview language
features at compile time.
* `-javacoption:-Xplugin:ValueClassPlugin` — activates the plugin.
The plugin scans each compilation unit after parsing and converts any class
annotated with `@jdk.test.lib.valueclass.AsValueClass` into a value class by
setting the internal `VALUE_CLASS` modifier flag and clearing the
`IDENTITY_TYPE` flag. This transformation only takes effect when
`--enable-preview` is active; without it the annotation is a no-op and the
class compiles as an ordinary identity class, so the same test source can
exercise both code paths.
Example:
$ make test TEST=jdk_lang JTREG="VALUE_CLASS_PLUGIN=true"
#### JVMTI_STRESS_AGENT
Executes JTReg tests with JVM TI stress agent. The stress agent is the part of

188
doc/value-class-preview.html Normal file
View File

@ -0,0 +1,188 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<title>Migration of JDK Classes to Value Classes</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
ul.task-list li input[type="checkbox"] {
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
</style>
<link rel="stylesheet" href="../make/data/docs-resources/resources/jdk-default.css" />
<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>
<!--[if lt IE 9]>
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
<![endif]-->
</head>
<body>
<header id="title-block-header">
<h1 class="title">Migration of JDK Classes to Value Classes</h1>
</header>
<nav id="TOC" role="doc-toc">
<ul>
<li><a href="#introduction" id="toc-introduction">Introduction</a></li>
<li><a href="#the-build-process" id="toc-the-build-process">The Build
Process</a>
<ul>
<li><a href="#the-custom-handling-for-the-value-objects-jep"
id="toc-the-custom-handling-for-the-value-objects-jep">The Custom
Handling for the Value Objects JEP</a></li>
<li><a href="#the-general-preview-source-to-binary-pipeline"
id="toc-the-general-preview-source-to-binary-pipeline">The General
Preview Source to Binary Pipeline</a></li>
<li><a href="#non-goals" id="toc-non-goals">Non-Goals</a></li>
</ul></li>
<li><a href="#testing" id="toc-testing">Testing</a></li>
<li><a href="#wrapper-class-caches"
id="toc-wrapper-class-caches">Wrapper Class Caches</a></li>
<li><a href="#editing-this-document"
id="toc-editing-this-document">Editing This Document</a></li>
</ul>
</nav>
<h2 id="introduction">Introduction</h2>
<p>The Value Objects feature introduces value objects and migrates
suitable classes to value classes. This means that when preview features
are enabled, different class files are used for the migrated classes in
the Java class library.</p>
<p>To accomplish this, a built JDK uses <em>preview-specific</em> files
in <code>META-INF/preview</code>, which overrides the regular files of
the same name. For example,
<code>META-INF/preview/java/lang/Integer.class</code> overrides
<code>java/lang/Integer.class</code>.</p>
<p>The JDK generates preview-specific source files (they may use preview
language features), compiles class files from them, and distributes
these class files in <code>META-INF/preview</code>.</p>
<h2 id="the-build-process">The Build Process</h2>
<h3 id="the-custom-handling-for-the-value-objects-jep">The Custom
Handling for the Value Objects JEP</h3>
<p>The Value Objects JEP requires a few select classes in the
<code>java.base</code> module to become value classes when preview
features are enabled.</p>
<p>The build of <code>java.base</code> module first creates the source
code of those value classes, done in <a
href="../make/modules/java.base/gensrc/GensrcValueClasses.gmk"><code>GensrcValueClasses.gmk</code></a>.</p>
<ol type="1">
<li><p>A hardcoded list of regular source files are selected for
preview-specific generation.</p></li>
<li><p>Extract the content of each regular source file, search for any
occurrences of <code>/*value*/ class</code> or
<code>/*value*/ record</code>, and replace with <code>value class</code>
or <code>value record</code>.</p></li>
<li><p>The replaced contents are written to the preview-specific
generated files, located in
<code>support/gensrc-valueclasses/java.base/</code>. The regular source
files remain unchanged in their original locations.</p></li>
<li><p>The general preview source to binary build pipeline recognizes
the <code>support/gensrc-valueclasses/java.base/</code> directory as
where the <code>java.base</code> module places its preview-specific
source files.</p></li>
</ol>
<h3 id="the-general-preview-source-to-binary-pipeline">The General
Preview Source to Binary Pipeline</h3>
<p>Once the preview-specific source files are ready, they are picked up
by the build system into a fully automated pipeline handling all modules
and all outcome images.</p>
<ol type="1">
<li><p>The <code>GENERATED_PREVIEW_SUBDIRS</code> variable in <a
href="../make/common/Modules.gmk"><code>make/common/Modules.gmk</code></a>
indicates where the source files are found.</p></li>
<li><p>For each module that has preview-specific source files, a goal is
created to compile these source files into class files.</p></li>
<li><p>The class files from each of these tasks reside in
<code>support/preview/&lt;module&gt;</code> for each module.</p></li>
<li><p>These preview-specific class files and other resources are copied
to the <code>META-INF/preview</code> directory of the regular output
directory.</p></li>
<li><p>At run-time, jimage will pick up the preview-specific overrides
from <code>META-INF/preview</code> only when preview features are
enabled.</p></li>
<li><p>The interim javac used by the build system cannot pick up the
preview-specific overrides; they must be supplied explicitly with the
following javac flag for every single module where overrides are
significant:</p>
<pre><code>--patch-module &lt;module&gt;=$(SUPPORT_OUTPUTDIR)/preview/&lt;module&gt;</code></pre>
<p>See <a
href="../make/test/BuildMicrobenchmark.gmk"><code>BuildMicroBenchmarks.gmk</code></a>
for an example.</p></li>
</ol>
<h3 id="non-goals">Non-Goals</h3>
<p>The Value Objects JEP only plans to introduce value classes that
are:</p>
<ol type="1">
<li><p>In the <code>java.base</code> module.</p>
<p>There's no plan to migrate other classes in other modules.</p></li>
<li><p>Migrated from existing classes.</p>
<p>These classes are available as identity classes when preview features
are disabled. There's no plan to introduce completely new value
classes.</p></li>
</ol>
<p>Support for other value classes would require significant changes to
the build system.</p>
<h2 id="testing">Testing</h2>
<p>In addition to tests that require preview features to be enabled,
tests that do not depend on preview features wish to run with preview
features enabled to ensure compatibility:</p>
<ol type="1">
<li><p>Some tests wish to run against the Java SE class library with
value classes.</p>
<p>The jtreg tests may be run with
<code>JTREG=VM_OPTIONS=--enable-preview</code>.</p></li>
<li><p>Some tests wish to run against their own classes migrated to
value classes.</p>
<p>The jtreg <a
href="testing.html#VALUE_CLASS_PLUGIN">VALUE_CLASS_PLUGIN</a> allows
tests to migrate their own classes to value classes when running with
the plugin.</p></li>
</ol>
<h2 id="wrapper-class-caches">Wrapper Class Caches</h2>
<p>Currently, wrapper class caches are retained even when preview
features are enabled to address performance losses. They have no
semantic impact to value objects.</p>
<p>In interpreter or C1 execution in Hotspot, allocations of a value
object to the heap as a full object with header happen when a value
object is:</p>
<ol type="1">
<li>Loaded from a flat storage (field or array)</li>
<li>Created by a constructor</li>
<li>If C2 uses scalarized calling convention, at C2 to C1/interpreter
calls and returns</li>
</ol>
<p>Ideally, C2 can eliminate such allocations, but this does not work if
the resulting object is stored into references. Unfortunately, many uses
of boxing conversions store the resulting wrapper objects as
references.</p>
<p>For the uses that store wrapper objects to references, if the boxing
conversion is:</p>
<ol type="1">
<li>Returning a value object from a flat cache array</li>
<li>Calling the value class constructor</li>
</ol>
<p>Then we would have heap allocation on every single use.</p>
<p>To avoid the allocations, we fall back to returning a value object
from a reference cache array, from which the loaded reference is
directly storable into a destination that wants a reference without any
allocation.</p>
<p>Since Hotspot may create flat arrays if an array of value objects is
requested by regular Java array creation mechanisms, we use
<code>ValueClass.newReferenceArray</code> to ensure we always create a
reference cache array.</p>
<p>The cache array for value objects may be removed without notice if
the performance losses from allocations are no longer significant.</p>
<h2 id="editing-this-document">Editing This Document</h2>
<p>If you want to contribute changes to this document, edit
<code>doc/value-class-preview.md</code> and then run
<code>make update-build-docs</code> to generate the same changes in
<code>doc/value-class-preview.html</code>.</p>
</body>
</html>

151
doc/value-class-preview.md Normal file
View File

@ -0,0 +1,151 @@
% Migration of JDK Classes to Value Classes
## Introduction
The Value Objects feature introduces value objects and migrates suitable classes
to value classes. This means that when preview features are enabled, different
class files are used for the migrated classes in the Java class library.
To accomplish this, a built JDK uses *preview-specific* files in `META-INF/preview`,
which overrides the regular files of the same name. For example,
`META-INF/preview/java/lang/Integer.class` overrides `java/lang/Integer.class`.
The JDK generates preview-specific source files (they may use preview language
features), compiles class files from them, and distributes these class files in
`META-INF/preview`.
## The Build Process
### The Custom Handling for the Value Objects JEP
The Value Objects JEP requires a few select classes in the `java.base` module
to become value classes when preview features are enabled.
The build of `java.base` module first creates the source code of those value
classes, done in
[`GensrcValueClasses.gmk`](../make/modules/java.base/gensrc/GensrcValueClasses.gmk).
1. A hardcoded list of regular source files are selected for preview-specific
generation.
2. Extract the content of each regular source file, search for any occurrences
of `/*value*/ class` or `/*value*/ record`, and replace with `value class`
or `value record`.
3. The replaced contents are written to the preview-specific generated files,
located in `support/gensrc-valueclasses/java.base/`. The regular source
files remain unchanged in their original locations.
4. The general preview source to binary build pipeline recognizes the
`support/gensrc-valueclasses/java.base/` directory as where the `java.base`
module places its preview-specific source files.
### The General Preview Source to Binary Pipeline
Once the preview-specific source files are ready, they are picked up by the
build system into a fully automated pipeline handling all modules and all
outcome images.
1. The `GENERATED_PREVIEW_SUBDIRS` variable in [`make/common/Modules.gmk`](../make/common/Modules.gmk)
indicates where the source files are found.
2. For each module that has preview-specific source files, a goal is created
to compile these source files into class files.
3. The class files from each of these tasks reside in `support/preview/<module>`
for each module.
4. These preview-specific class files and other resources are copied to the
`META-INF/preview` directory of the regular output directory.
5. At run-time, jimage will pick up the preview-specific overrides from
`META-INF/preview` only when preview features are enabled.
6. The interim javac used by the build system cannot pick up the
preview-specific overrides; they must be supplied explicitly with the
following javac flag for every single module where overrides are significant:
```
--patch-module <module>=$(SUPPORT_OUTPUTDIR)/preview/<module>
```
See [`BuildMicroBenchmarks.gmk`](../make/test/BuildMicrobenchmark.gmk) for an example.
### Non-Goals
The Value Objects JEP only plans to introduce value classes that are:
1. In the `java.base` module.
There's no plan to migrate other classes in other modules.
2. Migrated from existing classes.
These classes are available as identity classes when preview features are
disabled. There's no plan to introduce completely new value classes.
Support for other value classes would require significant changes to the build
system.
## Testing
In addition to tests that require preview features to be enabled, tests that do
not depend on preview features wish to run with preview features enabled to
ensure compatibility:
1. Some tests wish to run against the Java SE class library with value classes.
The jtreg tests may be run with `JTREG=VM_OPTIONS=--enable-preview`.
2. Some tests wish to run against their own classes migrated to value classes.
The jtreg [VALUE_CLASS_PLUGIN](testing.html#VALUE_CLASS_PLUGIN) allows tests
to migrate their own classes to value classes when running with the plugin.
## Wrapper Class Caches
Currently, wrapper class caches are retained even when preview features are
enabled to address performance losses. They have no semantic impact to value
objects.
In interpreter or C1 execution in Hotspot, allocations of a value object to the
heap as a full object with header happen when a value object is:
1. Loaded from a flat storage (field or array)
2. Created by a constructor
3. If C2 uses scalarized calling convention, at C2 to C1/interpreter calls and returns
Ideally, C2 can eliminate such allocations, but this does not work if the
resulting object is stored into references. Unfortunately, many uses of boxing
conversions store the resulting wrapper objects as references.
For the uses that store wrapper objects to references, if the boxing
conversion is:
1. Returning a value object from a flat cache array
2. Calling the value class constructor
Then we would have heap allocation on every single use.
To avoid the allocations, we fall back to returning a value object from a
reference cache array, from which the loaded reference is directly storable
into a destination that wants a reference without any allocation.
Since Hotspot may create flat arrays if an array of value objects is requested
by regular Java array creation mechanisms, we use `ValueClass.newReferenceArray`
to ensure we always create a reference cache array.
The cache array for value objects may be removed without notice if the
performance losses from allocations are no longer significant.
## Editing This Document
If you want to contribute changes to this document, edit `doc/value-class-preview.md`
and then run `make update-build-docs` to generate the same changes in
`doc/value-class-preview.html`.
---
# Override styles from the base CSS file that are not ideal for this document.
header-includes:
- '<style type="text/css">pre, code, tt { color: #1d6ae5; }</style>'
---

68
make/CompileJavaModules.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2014, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2014, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -30,6 +30,8 @@ include MakeFileStart.gmk
include JavaCompilation.gmk
include Modules.gmk
include CopyFiles.gmk
################################################################################
# If this is an imported module that has prebuilt classes, only compile
# module-info.java.
@ -98,13 +100,15 @@ endif
################################################################################
# Setup the main compilation
COMPILATION_OUTPUTDIR := $(if $($(MODULE)_BIN), $($(MODULE)_BIN), $(JDK_OUTPUTDIR)/modules)
$(eval $(call SetupJavaCompilation, $(MODULE), \
SMALL_JAVA := false, \
MODULE := $(MODULE), \
SRC := $(wildcard $(MODULE_SRC_DIRS)), \
INCLUDES := $(JDK_USER_DEFINED_FILTER), \
FAIL_NO_SRC := $(FAIL_NO_SRC), \
BIN := $(if $($(MODULE)_BIN), $($(MODULE)_BIN), $(JDK_OUTPUTDIR)/modules), \
BIN := $(COMPILATION_OUTPUTDIR), \
HEADERS := $(SUPPORT_OUTPUTDIR)/headers, \
CREATE_API_DIGEST := true, \
CLEAN := $(CLEAN), \
@ -126,6 +130,66 @@ $(eval $(call SetupJavaCompilation, $(MODULE), \
TARGETS += $($(MODULE))
################################################################################
# Setup compilation for preview classes in the module
# TBD: When $(DOCLINT) was included there was an NPE in JavacTypes.getOverriddenMethods
# Directory and file name suffix for jar file containing preview classes/resources.
PREVIEW_CLASSES_LABEL := preview
# Module relative path in which preview classes/resources are placed.
PREVIEW_PATH := META-INF/preview
MODULE_PREVIEW_SRC_DIRS := $(call FindModulePreviewSrcDirs, $(MODULE))
MODULE_PREVIEW_SOURCEPATH := $(call GetModulePreviewSrcPath)
ifneq ($(MODULE_PREVIEW_SRC_DIRS),)
# Compile preview classes into a separate directory, and then copy into the
# correct output path location. We cannot compile directly into the desired
# directory because it's the compiler which creates the original
# '<module>/<classpath>/...' hierarchy.
PREVIEW_OUTPUTDIR := $(SUPPORT_OUTPUTDIR)/$(PREVIEW_CLASSES_LABEL)
PATCH_COMMAND := $(MODULE)=$(call strip, $(COMPILATION_OUTPUTDIR)/$(MODULE))
$(eval $(call SetupJavaCompilation, $(MODULE)-$(PREVIEW_CLASSES_LABEL), \
SMALL_JAVA := false, \
MODULE := $(MODULE), \
SRC := $(wildcard $(MODULE_PREVIEW_SRC_DIRS)), \
INCLUDES := $(JDK_USER_DEFINED_FILTER), \
FAIL_NO_SRC := $(FAIL_NO_SRC), \
BIN := $(PREVIEW_OUTPUTDIR)/, \
DISABLED_WARNINGS := $(DISABLED_WARNINGS_java) preview, \
EXCLUDES := $(EXCLUDES), \
EXCLUDE_FILES := $(EXCLUDE_FILES), \
KEEP_ALL_TRANSLATIONS := $(KEEP_ALL_TRANSLATIONS), \
DEPENDS := $($(MODULE)), \
JAVAC_FLAGS := \
$(JAVAC_FLAGS) \
--module-source-path $(MODULE_PREVIEW_SOURCEPATH) \
--module-path $(MODULEPATH) \
--patch-module $(PATCH_COMMAND) \
--system none \
--enable-preview -source $(JDK_SOURCE_TARGET_VERSION), \
))
# Don't add '$($(MODULE)-$(PREVIEW_CLASSES_LABEL))' to TARGETS (it's transient).
# The 'preview' target below depends on it, and that's the non-transient
# result we care about.
# Copy compiled output from "$(PREVIEW_OUTPUTDIR)/$(MODULE)/<classpath>/..."
# to "$(COMPILATION_OUTPUTDIR)/$(MODULE)/$(PREVIEW_PATH)/<classpath>/...".
MOD_SRC := $(PREVIEW_OUTPUTDIR)/$(MODULE)
MOD_DST := $(COMPILATION_OUTPUTDIR)/$(MODULE)
# NOTE: We cannot use '$(CP) -R $(MOD_SRC)/*/ ...' to select sub-directories (it
# does not work on MacOS/BSD). Use 'filter-out' to explicitly exclude marker files.
$(MOD_DST)/_the.$(MODULE).preview: $($(MODULE)-$(PREVIEW_CLASSES_LABEL))
$(RM) -r $(@D)/$(PREVIEW_PATH)
$(MKDIR) -p $(@D)/$(PREVIEW_PATH)
$(CP) -R $(filter-out $(MOD_SRC)/_%, $(wildcard $(MOD_SRC)/*)) $(@D)/$(PREVIEW_PATH)
$(TOUCH) $@
TARGETS += $(MOD_DST)/_the.$(MODULE).preview
endif
# Declare dependencies between java compilations of different modules.
# Since the other modules are declared in different invocations of this file,
# use the macro to find the correct target file to depend on.

3
make/Docs.gmk
View File

@ -93,9 +93,10 @@ JAVADOC_DISABLED_DOCLINT_WARNINGS := missing
JAVADOC_DISABLED_DOCLINT_PACKAGES := org.w3c.* javax.smartcardio
# The initial set of options for javadoc
JAVADOC_OPTIONS := -use -keywords -notimestamp \
JAVADOC_OPTIONS := -XDignore.symbol.file=true -use -keywords -notimestamp \
-serialwarn -encoding utf-8 -docencoding utf-8 -breakiterator \
-splitIndex --system none -javafx --expand-requires transitive \
--enable-preview -source $(JDK_SOURCE_TARGET_VERSION) \
--override-methods=summary
# The reference options must stay stable to allow for comparisons across the

52
make/Images.gmk
View File

@ -138,22 +138,22 @@ CDS_DUMP_FLAGS = -Xmx128M -Xms128M
#
# Param1 - VM variant (e.g., server, client, zero, ...)
# Param2 - _nocoops, _nocoh, _nocoops_nocoh, or empty
# Param3 - _preview, or empty
define CreateCDSArchive
$1_$2_COOPS_OPTION := $(if $(findstring _nocoops, $2),-XX:-UseCompressedOops)
# enable and also explicitly disable coh as needed.
ifeq ($(call isTargetCpuBits, 64), true)
$1_$2_NOCOH_OPTION := -XX:+UnlockExperimentalVMOptions \
$(if $(findstring _nocoh, $2),-XX:-UseCompactObjectHeaders,-XX:+UseCompactObjectHeaders)
$1_$2_NOCOH_OPTION := $(if $(findstring _nocoh, $2),-XX:-UseCompactObjectHeaders,-XX:+UseCompactObjectHeaders)
endif
$1_$2_DUMP_EXTRA_ARG := $$($1_$2_COOPS_OPTION) $$($1_$2_NOCOH_OPTION)
$1_$2_DUMP_TYPE := $(if $(findstring _nocoops, $2),-NOCOOPS,)$(if $(findstring _nocoh, $2),-NOCOH,)
$1_$2_$3_DUMP_EXTRA_ARG := $$($1_$2_COOPS_OPTION) $$($1_$2_NOCOH_OPTION) $(if $(findstring _preview, $3), --enable-preview,)
$1_$2_$3_DUMP_TYPE := $(if $(findstring _nocoops, $2),-NOCOOPS,)$(if $(findstring _nocoh, $2),-NOCOH,)$(if $(findstring _preview, $3),-PREVIEW,)
$1_$2_CDS_DUMP_FLAGS := $(CDS_DUMP_FLAGS) $(if $(filter g1gc, $(JVM_FEATURES_$1)), -XX:+UseG1GC)
$1_$2_$3_CDS_DUMP_FLAGS := $(CDS_DUMP_FLAGS) $(if $(filter g1gc, $(JVM_FEATURES_$1)), -XX:+UseG1GC)
ifeq ($(OPENJDK_TARGET_OS), windows)
$1_$2_CDS_ARCHIVE := bin/$1/classes$2.jsa
$1_$2_$3_CDS_ARCHIVE := bin/$1/classes$2$3.jsa
else
$1_$2_CDS_ARCHIVE := lib/$1/classes$2.jsa
$1_$2_$3_CDS_ARCHIVE := lib/$1/classes$2$3.jsa
endif
ifneq ($(COMPARE_BUILD), )
@ -161,51 +161,55 @@ define CreateCDSArchive
endif
ifeq ($(DEBUG_CDS_ARCHIVE), true)
$1_$2_CDS_DUMP_FLAGS += -Xlog:aot+map*=trace:file=$$(JDK_IMAGE_DIR)/$$($1_$2_CDS_ARCHIVE).cdsmap:none:filesize=0
$1_$2_$3_CDS_DUMP_FLAGS += -Xlog:aot+map*=trace:file=$$(JDK_IMAGE_DIR)/$$($1_$2_$3_CDS_ARCHIVE).cdsmap:none:filesize=0
endif
$$(eval $$(call SetupExecute, $1_$2_gen_cds_archive_jdk, \
WARN := Creating CDS$$($1_$2_DUMP_TYPE) archive for jdk image for $1, \
INFO := Using CDS flags for $1: $$($1_$2_CDS_DUMP_FLAGS), \
$$(eval $$(call SetupExecute, $1_$2_$3_gen_cds_archive_jdk, \
WARN := Creating CDS$$($1_$2_$3_DUMP_TYPE) archive for jdk image for $1, \
INFO := Using CDS flags for $1: $$($1_$2_$3_CDS_DUMP_FLAGS), \
DEPS := $$(jlink_jdk), \
OUTPUT_FILE := $$(JDK_IMAGE_DIR)/$$($1_$2_CDS_ARCHIVE), \
OUTPUT_FILE := $$(JDK_IMAGE_DIR)/$$($1_$2_$3_CDS_ARCHIVE), \
SUPPORT_DIR := $$(JDK_IMAGE_SUPPORT_DIR), \
COMMAND := $$(FIXPATH) $$(JDK_IMAGE_DIR)/bin/java -Xshare:dump \
-XX:SharedArchiveFile=$$(JDK_IMAGE_DIR)/$$($1_$2_CDS_ARCHIVE) \
-$1 $$($1_$2_DUMP_EXTRA_ARG) $$($1_$2_CDS_DUMP_FLAGS) $$(LOG_INFO), \
-XX:SharedArchiveFile=$$(JDK_IMAGE_DIR)/$$($1_$2_$3_CDS_ARCHIVE) \
-$1 $$($1_$2_$3_DUMP_EXTRA_ARG) $$($1_$2_$3_CDS_DUMP_FLAGS) $$(LOG_INFO), \
))
JDK_TARGETS += $$($1_$2_gen_cds_archive_jdk)
JDK_TARGETS += $$($1_$2_$3_gen_cds_archive_jdk)
$$(eval $$(call SetupExecute, $1_$2_gen_cds_archive_jre, \
WARN := Creating CDS$$($1_$2_DUMP_TYPE) archive for jre image for $1, \
INFO := Using CDS flags for $1: $$($1_$2_CDS_DUMP_FLAGS), \
$$(eval $$(call SetupExecute, $1_$2_$3_gen_cds_archive_jre, \
WARN := Creating CDS$$($1_$2_$3_DUMP_TYPE) archive for jre image for $1, \
INFO := Using CDS flags for $1: $$($1_$2_$3_CDS_DUMP_FLAGS), \
DEPS := $$(jlink_jre), \
OUTPUT_FILE := $$(JRE_IMAGE_DIR)/$$($1_$2_CDS_ARCHIVE), \
OUTPUT_FILE := $$(JRE_IMAGE_DIR)/$$($1_$2_$3_CDS_ARCHIVE), \
SUPPORT_DIR := $$(JRE_IMAGE_SUPPORT_DIR), \
COMMAND := $$(FIXPATH) $$(JRE_IMAGE_DIR)/bin/java -Xshare:dump \
-XX:SharedArchiveFile=$$(JRE_IMAGE_DIR)/$$($1_$2_CDS_ARCHIVE) \
-$1 $$($1_$2_DUMP_EXTRA_ARG) $$($1_$2_CDS_DUMP_FLAGS) $$(LOG_INFO), \
-XX:SharedArchiveFile=$$(JRE_IMAGE_DIR)/$$($1_$2_$3_CDS_ARCHIVE) \
-$1 $$($1_$2_$3_DUMP_EXTRA_ARG) $$($1_$2_$3_CDS_DUMP_FLAGS) $$(LOG_INFO), \
))
JRE_TARGETS += $$($1_$2_gen_cds_archive_jre)
JRE_TARGETS += $$($1_$2_$3_gen_cds_archive_jre)
endef
ifeq ($(BUILD_CDS_ARCHIVE), true)
$(foreach v, $(JVM_VARIANTS), \
$(eval $(call CreateCDSArchive,$v,)) \
$(eval $(call CreateCDSArchive,$v,,)) \
$(eval $(call CreateCDSArchive,$v,,_preview)) \
)
ifeq ($(call isTargetCpuBits, 64), true)
$(foreach v, $(JVM_VARIANTS), \
$(eval $(call CreateCDSArchive,$v,_nocoops)) \
$(eval $(call CreateCDSArchive,$v,_nocoops,)) \
$(eval $(call CreateCDSArchive,$v,_nocoops,_preview)) \
)
ifeq ($(BUILD_CDS_ARCHIVE_NOCOH), true)
$(foreach v, $(JVM_VARIANTS), \
$(eval $(call CreateCDSArchive,$v,_nocoh)) \
$(eval $(call CreateCDSArchive,$v,_nocoh,_preview)) \
)
$(foreach v, $(JVM_VARIANTS), \
$(eval $(call CreateCDSArchive,$v,_nocoops_nocoh)) \
$(eval $(call CreateCDSArchive,$v,_nocoops_nocoh,_preview)) \
)
endif
endif

16
make/Main.gmk
View File

@ -794,6 +794,20 @@ ifeq ($(BUILD_JTREG_TEST_THREAD_FACTORY), true)
))
endif
# Builds the value class plugin jtreg extension (JEP 401)
$(eval $(call SetupTarget, build-test-value-class-plugin, \
MAKEFILE := test/BuildJtregValueClassPlugin, \
TARGET := build, \
DEPS := interim-langtools exploded-image, \
))
# Copies the value class plugin into the test image
$(eval $(call SetupTarget, test-image-value-class-plugin, \
MAKEFILE := test/BuildJtregValueClassPlugin, \
TARGET := images, \
DEPS := build-test-value-class-plugin, \
))
$(eval $(call SetupTarget, build-microbenchmark, \
MAKEFILE := test/BuildMicrobenchmark, \
DEPS := interim-langtools exploded-image build-test-lib, \
@ -1310,6 +1324,8 @@ ifeq ($(BUILD_JTREG_TEST_THREAD_FACTORY), true)
test-image: test-image-test-thread-factory
endif
test-image: test-image-value-class-plugin
ifneq ($(JMH_CORE_JAR), )
test-image: build-microbenchmark
endif

3
make/MainSupport.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -107,6 +107,7 @@ define Clean-java
@$(PRINTF) "Cleaning java %s..." "$(if $1,for $(strip $1) )"
@$(ECHO) "" $(LOG_DEBUG)
$(RM) -r $(JDK_OUTPUTDIR)/modules/$(strip $1)
$(RM) -r $(SUPPORT_OUTPUTDIR)/preview/$(strip $1)
$(RM) -r $(SUPPORT_OUTPUTDIR)/special_classes/$(strip $1)
$(ECHO) " done"
$(PRINTF) "Cleaning headers %s..." "$(if $1,for $(strip $1) )"

19
make/RunTests.gmk
View File

@ -93,6 +93,9 @@ JTREG_FAILURE_HANDLER := $(JTREG_FAILURE_HANDLER_DIR)/jtregFailureHandler.jar
JTREG_TEST_THREAD_FACTORY_DIR := $(TEST_IMAGE_DIR)/jtreg_test_thread_factory
JTREG_TEST_THREAD_FACTORY_JAR := $(JTREG_TEST_THREAD_FACTORY_DIR)/jtregTestThreadFactory.jar
JTREG_VALUE_CLASS_PLUGIN_DIR := $(TEST_IMAGE_DIR)/jtreg_value_class_plugin
JTREG_VALUE_CLASS_PLUGIN_JAR := $(JTREG_VALUE_CLASS_PLUGIN_DIR)/valueClassPlugin.jar
JTREG_FAILURE_HANDLER_TIMEOUT ?= 0
ifneq ($(wildcard $(JTREG_FAILURE_HANDLER)), )
@ -206,7 +209,7 @@ $(eval $(call ParseKeywordVariable, JTREG, \
SINGLE_KEYWORDS := JOBS TIMEOUT_FACTOR FAILURE_HANDLER_TIMEOUT \
TEST_MODE ASSERT VERBOSE RETAIN TEST_THREAD_FACTORY JVMTI_STRESS_AGENT \
MAX_MEM RUN_PROBLEM_LISTS RETRY_COUNT REPEAT_COUNT MAX_OUTPUT REPORT \
AOT_JDK MANUAL $(CUSTOM_JTREG_SINGLE_KEYWORDS), \
AOT_JDK MANUAL VALUE_CLASS_PLUGIN $(CUSTOM_JTREG_SINGLE_KEYWORDS), \
STRING_KEYWORDS := OPTIONS JAVA_OPTIONS VM_OPTIONS KEYWORDS \
EXTRA_PROBLEM_LISTS LAUNCHER_OPTIONS \
$(CUSTOM_JTREG_STRING_KEYWORDS), \
@ -877,6 +880,20 @@ define SetupRunJtregTestBody
))
endif
ifneq ($$(JTREG_VALUE_CLASS_PLUGIN), )
ifneq ($$(wildcard $$(JTREG_VALUE_CLASS_PLUGIN_JAR)), )
$1_JTREG_BASIC_OPTIONS += -cpa:$$(JTREG_VALUE_CLASS_PLUGIN_JAR)
endif
$1_JTREG_BASIC_OPTIONS += -vmoption:--enable-preview
$1_JTREG_BASIC_OPTIONS += -javacoption:-XDaccessInternalAPI
$1_JTREG_BASIC_OPTIONS += -javacoption:--source -javacoption:$(VERSION_FEATURE)
$1_JTREG_BASIC_OPTIONS += -javacoption:--enable-preview
$1_JTREG_BASIC_OPTIONS += -javacoption:-Xplugin:ValueClassPlugin
$1_JTREG_BASIC_OPTIONS += $$(addprefix $$(JTREG_PROBLEM_LIST_PREFIX), $$(wildcard \
$$(addprefix $$($1_TEST_ROOT)/, ProblemList-ValueClass.txt) \
))
endif
ifneq ($$(JTREG_JVMTI_STRESS_AGENT), )
AGENT := $$(LIBRARY_PREFIX)JvmtiStressAgent$$(SHARED_LIBRARY_SUFFIX)=$$(JTREG_JVMTI_STRESS_AGENT)
$1_JTREG_BASIC_OPTIONS += -javaoption:'-agentpath:$(TEST_IMAGE_DIR)/hotspot/jtreg/native/$$(AGENT)'

3
make/RunTestsPrebuiltSpec.gmk
View File

@ -53,6 +53,9 @@ $(eval $(call VerifyVariable,TEST_IMAGE_DIR))
$(eval $(call VerifyVariable,MAKE))
$(eval $(call VerifyVariable,BASH))
include $(TOPDIR)/make/conf/version-numbers.conf
VERSION_FEATURE := $(DEFAULT_VERSION_FEATURE)
################################################################################
# The "human readable" name of this configuration
CONF_NAME := run-test-prebuilt

5
make/ToolsJdk.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -81,6 +81,9 @@ TOOL_GENERATEEXTRAPROPERTIES = $(JAVA_SMALL) -cp $(BUILDTOOLS_OUTPUTDIR)/jdk_too
TOOL_GENERATECASEFOLDING = $(JAVA_SMALL) -cp $(BUILDTOOLS_OUTPUTDIR)/jdk_tools_classes \
build.tools.generatecharacter.GenerateCaseFolding
TOOL_GENERATESPECIALCASING = $(JAVA_SMALL) -cp $(BUILDTOOLS_OUTPUTDIR)/jdk_tools_classes \
build.tools.generatespecialcasing.GenerateSpecialCasing
TOOL_MAKEZIPREPRODUCIBLE = $(JAVA_SMALL) -cp $(BUILDTOOLS_OUTPUTDIR)/jdk_tools_classes \
build.tools.makezipreproducible.MakeZipReproducible

59
make/ZipSource.gmk
View File

@ -35,12 +35,35 @@ $(if $(filter $(TOPDIR)/%, $(SUPPORT_OUTPUTDIR)), $(eval SRC_ZIP_BASE := $(TOPDI
################################################################################
# Create the directory structure for src.zip using symlinks.
# <module>/<package>/<file>.java
# <module>/<package>/<file>.java
# or:
# <module>/META-INF/preview/<package>/<file>.java
# for preview enabled sources.
#
# Generate the src dirs in the first make invocation and then call this makefile
# again to create src.zip.
ALL_MODULES := $(FindAllModules)
# Generate the src dirs in the first make invocation and then call this makefile
# again to create src.zip.
# Module source directories ($d) are things like (but not limited to):
# - $TOPDIR/src/<module>/share/classes
# - $SUPPORT_OUTPUTDIR/gensrc/<module>
#
# A symbolic link is created for each directory. For example:
# - $TOPDIR/src/<module>/share/classes
# is linked from:
# - $SRC_ZIP_WORK_DIR/src/<module>/share/classes/<module>
#
# Each link has the form:
# - $SRC_ZIP_WORK_DIR/<link-dir>/<module>
# and always has the name of its module (even if the module already appears in <link-dir>.
#
# Then the contents of all link parent directories are given as ZIP includes:
# - $SRC_ZIP_WORK_DIR/<link-dir>
# results in ZIP file entries starting:
# - <module>/...
# where multiple source links/directories can contribute to the same module's sources.
#
$(foreach m, $(ALL_MODULES), \
$(foreach d, $(call FindModuleSrcDirs, $m), \
$(eval $d_TARGET := $(SRC_ZIP_WORK_DIR)/$(patsubst $(TOPDIR)/%,%,$(patsubst $(SUPPORT_OUTPUTDIR)/%,%,$d))/$m) \
@ -54,6 +77,36 @@ $(foreach m, $(ALL_MODULES), \
) \
)
# Preview source directories are currently limited to:
# - $SUPPORT_OUTPUTDIR/gensrc-valueclasses/<module>
# If this is changed, the 'patsubst' rewriting will need updating.
#
# The difficulty with this case is that we need to have a directory of the form:
# - $SRC_ZIP_WORK_DIR/<link-dir>/<module>
# containing (either directly or via symbolic linking) a `META-INF/preview`
# subdirectory which then contains the sources.
#
# The easiest way to achieve this is to symbolically link:
# - $SRC_ZIP_WORK_DIR/<link-dir>/<module>/META-INF/preview
# and then use the parent directory:
# - $SRC_ZIP_WORK_DIR/<link-dir>/<module>
# as the actual target for the ZIP file generation. However, this requires
# an extra rule to associate the symbolic link and the parent directory.
#
$(foreach m, $(ALL_MODULES), \
$(foreach d, $(call FindModulePreviewSrcDirs, $m), \
$(eval $d_TARGET := $(SRC_ZIP_WORK_DIR)/$(patsubst $(SUPPORT_OUTPUTDIR)/%,%,$d)/$m) \
$(if $(SRC_GENERATED), , \
$(eval $$($d_TARGET)/META-INF/preview: $d ; \
$$(if $(filter $(SRC_ZIP_BASE)/%, $d), $$(link-file-relative), $$(link-file-absolute)) \
) \
$(eval $$($d_TARGET): $$($d_TARGET)/META-INF/preview) \
) \
$(eval SRC_ZIP_SRCS += $$($d_TARGET)) \
$(eval SRC_ZIP_SRCS_$m += $$($d_TARGET)) \
) \
)
TARGETS += $(SRC_ZIP_SRCS)
################################################################################

10
make/autoconf/basic.m4
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -327,14 +327,6 @@ AC_DEFUN_ONCE([BASIC_SETUP_DEVKIT],
elif test -d "$DEVKIT_ROOT/$host/sys-root"; then
SYSROOT="$DEVKIT_ROOT/$host/sys-root"
fi
if test "x$DEVKIT_ROOT" != x; then
DEVKIT_LIB_DIR="$DEVKIT_ROOT/lib"
if test "x$OPENJDK_TARGET_CPU_BITS" = x64; then
DEVKIT_LIB_DIR="$DEVKIT_ROOT/lib64"
fi
AC_SUBST(DEVKIT_LIB_DIR)
fi
fi
# You can force the sysroot if the sysroot encoded into the compiler tools

1
make/autoconf/flags-ldflags.m4
View File

@ -81,6 +81,7 @@ AC_DEFUN([FLAGS_SETUP_LDFLAGS_HELPER],
fi
if test "x$ENABLE_LINKTIME_GC" = xtrue; then
BASIC_LDFLAGS_JVM_ONLY="$BASIC_LDFLAGS_JVM_ONLY -Wl,--gc-sections -Wl,--undefined=_ZTV8Metadata"
BASIC_LDFLAGS_JDK_ONLY="$BASIC_LDFLAGS_JDK_ONLY -Wl,--gc-sections"
fi
fi

7
make/autoconf/jdk-options.m4
View File

@ -95,6 +95,13 @@ AC_DEFUN_ONCE([JDKOPT_SETUP_OPEN_OR_CUSTOM],
AC_DEFUN_ONCE([JDKOPT_SETUP_JDK_OPTIONS],
[
# Should we build a JDK with/without sound ?
UTIL_ARG_ENABLE(NAME: jsound, DEFAULT: true,
RESULT: ENABLE_JSOUND,
DESC: [build with sound support],
CHECKING_MSG: [if we should build with sound support])
AC_SUBST(ENABLE_JSOUND)
# Should we build a JDK without a graphical UI?
UTIL_ARG_ENABLE(NAME: headless-only, DEFAULT: false,
RESULT: ENABLE_HEADLESS_ONLY,

2
make/autoconf/lib-tests.m4
View File

@ -28,7 +28,7 @@
################################################################################
# Minimum supported versions
JTREG_MINIMUM_VERSION=8.2.1
JTREG_MINIMUM_VERSION=8.3
GTEST_MINIMUM_VERSION=1.14.0
################################################################################

4
make/autoconf/libraries.m4
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -149,7 +149,7 @@ AC_DEFUN_ONCE([LIB_SETUP_LIBRARIES],
if test "x$OPENJDK_TARGET_OS" = xwindows; then
BASIC_JVM_LIBS="$BASIC_JVM_LIBS kernel32.lib user32.lib gdi32.lib winspool.lib \
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib powrprof.lib uuid.lib \
ws2_32.lib winmm.lib version.lib psapi.lib"
ws2_32.lib winmm.lib version.lib"
fi
LIB_SETUP_JVM_LIBS(BUILD)
LIB_SETUP_JVM_LIBS(TARGET)

2
make/autoconf/spec.gmk.template
View File

@ -329,6 +329,8 @@ HOTSPOT_OVERRIDE_LIBPATH := @HOTSPOT_OVERRIDE_LIBPATH@
# Control use of precompiled header in hotspot libjvm build
USE_PRECOMPILED_HEADER := @USE_PRECOMPILED_HEADER@
ENABLE_JSOUND := @ENABLE_JSOUND@
# Only build headless support or not
ENABLE_HEADLESS_ONLY := @ENABLE_HEADLESS_ONLY@

20
make/common/Modules.gmk
View File

@ -68,6 +68,12 @@ GENERATED_SRC_DIRS += \
$(SUPPORT_OUTPUTDIR)/gensrc \
#
# Directories in which generated preview classes may exist.
# Currently this is restricted to generated value classes, but can be extended.
GENERATED_PREVIEW_SUBDIRS += \
$(SUPPORT_OUTPUTDIR)/gensrc-valueclasses \
#
TOP_SRC_DIRS += \
$(TOPDIR)/src \
#
@ -137,6 +143,13 @@ FindModuleSrcDirs = \
$(addsuffix /$(strip $1), $(GENERATED_SRC_DIRS) $(IMPORT_MODULES_SRC)) \
$(foreach sub, $(SRC_SUBDIRS), $(addsuffix /$(strip $1)/$(sub), $(TOP_SRC_DIRS)))))
# Find preview class source dirs for a particular module.
# Currently this is restricted to generated value classes, but can be extended.
# $1 - Module to find source dirs for
FindModulePreviewSrcDirs = \
$(strip $(wildcard \
$(addsuffix /$(strip $1), $(GENERATED_PREVIEW_SUBDIRS))))
# Find all specs dirs for a particular module
# $1 - Module to find specs dirs for
FindModuleSpecsDirs = \
@ -159,6 +172,13 @@ GetModuleSrcPath = \
$(addsuffix /*, $(GENERATED_SRC_DIRS) $(IMPORT_MODULES_SRC)) \
$(foreach sub, $(SRC_SUBDIRS), $(addsuffix /*/$(sub), $(TOP_SRC_DIRS))))
# Construct the complete module source path for preview classes.
# Currently this is restricted to generated value classes, but can be extended.
GetModulePreviewSrcPath = \
$(call PathList, \
$(addsuffix /*, $(GENERATED_PREVIEW_SUBDIRS) $(GENERATED_SRC_DIRS) $(IMPORT_MODULES_SRC)) \
$(foreach sub, $(SRC_SUBDIRS), $(addsuffix /*/$(sub), $(TOP_SRC_DIRS))))
################################################################################
# Extract module dependencies from module-info.java files, both normal
# dependencies ("requires"), and indirect exports ("requires transitive").

9
make/common/modules/GensrcStreamPreProcessing.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2025, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -33,9 +33,12 @@ ifeq ($(INCLUDE), true)
include Execute.gmk
include $(TOPDIR)/make/ToolsJdk.gmk
NON_BYTE_NUMBER_TYPES := char short int long float double
NON_BYTE_INTEGER_TYPES := char short int long
NON_BYTE_NUMBER_TYPES := $(NON_BYTE_INTEGER_TYPES) float double
NUMBER_TYPES := byte $(NON_BYTE_NUMBER_TYPES)
INTEGER_NUMBER_TYPES := byte $(NON_BYTE_INTEGER_TYPES)
PRIMITIVE_TYPES := boolean $(NUMBER_TYPES)
BITWISE_PRIMITIVE_TYPES := boolean $(INTEGER_NUMBER_TYPES)
################################################################################
# The Conv function converts a type given as first argument (as a normal Java
@ -116,7 +119,7 @@ Conv_A = \
# Return integer type with same size as the type
Conv_memtype = \
$(if $(filter float, $1), int, $(if $(filter double, $1), long, $1))
$(if $(filter float, $1), int, $(if $(filter double, $1), long, $(if $(filter boolean, $1), byte, $1)))
# Return capitalized integer type with same size as the type
Conv_Memtype = \

2
make/conf/github-actions.conf
View File

@ -26,7 +26,7 @@
# Versions and download locations for dependencies used by GitHub Actions (GHA)
GTEST_VERSION=1.14.0
JTREG_VERSION=8.2.1+1
JTREG_VERSION=8.3+1
LINUX_X64_BOOT_JDK_EXT=tar.gz
LINUX_X64_BOOT_JDK_URL=https://download.java.net/java/GA/jdk26/c3cc523845074aa0af4f5e1e1ed4151d/35/GPL/openjdk-26_linux-x64_bin.tar.gz

6
make/conf/jib-profiles.js
View File

@ -644,7 +644,7 @@ var getJibProfilesProfiles = function (input, common, data) {
// Bootcycle profiles runs the build with itself as the boot jdk. This can
// be done in two ways. Either using the builtin bootcycle target in the
// build system. Or by supplying the main jdk build as bootjdk to configure.
[ "linux-x64", "macosx-x64", "windows-x64", "linux-aarch64" ]
[ "linux-x64", "macosx-aarch64", "macosx-x64", "windows-x64", "linux-aarch64" ]
.forEach(function (name) {
var bootcycleName = name + "-bootcycle";
var bootcyclePrebuiltName = name + "-bootcycle-prebuilt";
@ -1174,9 +1174,9 @@ var getJibProfilesDependencies = function (input, common) {
jtreg: {
server: "jpg",
product: "jtreg",
version: "8.2.1",
version: "8.3",
build_number: "1",
file: "bundles/jtreg-8.2.1+1.zip",
file: "bundles/jtreg-8.3+1.zip",
environment_name: "JT_HOME",
environment_path: input.get("jtreg", "home_path") + "/bin",
configure_args: "--with-jtreg=" + input.get("jtreg", "home_path"),

3
make/hotspot/lib/JvmFeatures.gmk
View File

@ -52,7 +52,8 @@ ifeq ($(call check-jvm-feature, zero), true)
JVM_EXCLUDES += opto libadt
JVM_EXCLUDE_PATTERNS += c1_ c1/ c2_ runtime_ /c2/
JVM_EXCLUDE_FILES += templateInterpreter.cpp \
templateInterpreterGenerator.cpp bcEscapeAnalyzer.cpp ciTypeFlow.cpp
templateInterpreterGenerator.cpp bcEscapeAnalyzer.cpp ciTypeFlow.cpp \
macroAssembler_common.cpp
JVM_CFLAGS_FEATURES += -DZERO \
-DZERO_LIBARCH='"$(OPENJDK_TARGET_CPU_LEGACY_LIB)"' $(LIBFFI_CFLAGS)
JVM_LIBS_FEATURES += $(LIBFFI_LIBS)

40
make/jdk/src/classes/build/tools/cldrconverter/ResourceBundleGenerator.java
View File

@ -29,6 +29,7 @@ import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.Comparator;
import java.util.Formatter;
import java.util.HashSet;
import java.util.HashMap;
@ -39,6 +40,7 @@ import java.util.Objects;
import java.util.Set;
import java.util.SortedSet;
import java.util.stream.Collectors;
import static java.util.ResourceBundle.Control;
class ResourceBundleGenerator implements BundleGenerator {
// preferred timezones - keeping compatibility with JDK1.1 3 letter abbreviations
@ -69,6 +71,9 @@ class ResourceBundleGenerator implements BundleGenerator {
// For duplicated values
private static final String META_VALUE_PREFIX = "metaValue_";
// locales in the base module
private final Set<Locale> baseModuleLocales = new HashSet<>();
@Override
public void generateBundle(String packageName, String baseName, String localeID,
Map<String, ?> map, BundleType type) throws IOException {
@ -80,8 +85,15 @@ class ResourceBundleGenerator implements BundleGenerator {
return;
}
// Assume that non-base resources go into jdk.localedata
if (!CLDRConverter.isBaseModule) {
if (CLDRConverter.isBaseModule) {
if (!localeID.equals("root")) {
baseModuleLocales.addAll(
Control.getControl(Control.FORMAT_DEFAULT)
.getCandidateLocales("",
Locale.forLanguageTag(CLDRConverter.toLanguageTag(localeID))));
}
} else {
// Assume that non-base resources go into jdk.localedata
dirName = dirName + File.separator + "ext";
packageName = packageName + ".ext";
}
@ -284,6 +296,7 @@ class ResourceBundleGenerator implements BundleGenerator {
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import sun.util.locale.provider.LocaleDataMetaInfo;
import sun.util.locale.provider.LocaleProviderAdapter;
@ -296,6 +309,7 @@ class ResourceBundleGenerator implements BundleGenerator {
out.printf("""
private static final Map<Locale, String[]> parentLocalesMap = HashMap.newHashMap(%d);
private static final Map<String, String> languageAliasMap = HashMap.newHashMap(%d);
private static final Set<Locale> baseModuleLocales;
static final boolean nonlikelyScript = %s; // package access from CLDRLocaleProviderAdapter
static {
@ -322,7 +336,23 @@ class ResourceBundleGenerator implements BundleGenerator {
CLDRConverter.handlerSupplMeta.getLanguageAliasData().forEach((key, value) -> {
out.printf(" languageAliasMap.put(\"%s\", \"%s\");\n", CLDRConverter.escape(key), CLDRConverter.escape(value));
});
out.printf(" }\n\n");
out.println();
// for baseModuleLocales
out.printf(" baseModuleLocales = Set.of(\n");
out.printf(" %s",
baseModuleLocales.stream()
.map(Locale::toLanguageTag)
.sorted(Comparator.comparing(l -> l.equals("und") ? "" : l))
.map(l -> switch(l) {
case "und" -> "Locale.ROOT";
case "en" -> "Locale.ENGLISH";
case "en-US" -> "Locale.US";
default -> "Locale.forLanguageTag(\"" + l + "\")";
})
.collect(Collectors.joining(",\n ")));
out.printf("\n );");
out.println("\n }\n");
// end of static initializer block.
@ -391,6 +421,10 @@ class ResourceBundleGenerator implements BundleGenerator {
return parentLocalesMap;
}
public Set<Locale> baseModuleLocales() {
return baseModuleLocales;
}
// package access from CLDRLocaleProviderAdapter
Map<String, String> likelyScriptMap() {
return CLDRMapHolder.likelyScriptMap;

100
make/jdk/src/classes/build/tools/generatespecialcasing/GenerateSpecialCasing.java Normal file
View File

@ -0,0 +1,100 @@
/*
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation. Oracle designates this
* particular file as subject to the "Classpath" exception as provided
* by Oracle in the LICENSE file that accompanied this code.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/
package build.tools.generatespecialcasing;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.Stream;
/**
* Parses UCD's "SpecialCasing.txt" file and extract selected special case
* mappings, then reformats each entry into `Entry` instances in
* `java.lang.ConditionalSpecialCasing` class.
*
* Arguments to this utility:
* args[0]: Full path to the "ConditionalSpecialCasing" template file
* args[1]: Full path to the "SpecialCasing.txt" UCD file
* args[2]: Full path to the generated output file
*/
public class GenerateSpecialCasing {
// Represents a code point with selected special casing.
private record Entry(String codePoint, List<String> lowerCase,
List<String> upperCase, String language,
String condition) {};
public static void main(String[] args) throws IOException {
var templateFile = Paths.get(args[0]);
var specialCasingFile = Paths.get(args[1]);
var gensrcFile = Paths.get(args[2]);
List<Entry> entries = Files.lines(specialCasingFile)
.filter(Predicate.not(l -> l.startsWith("#") || l.isBlank()))
.map(l -> l.replaceFirst("#.*$", "").split(";"))
// "U+0130" (LATIN_CAPITAL_LETTER_I_WITH_DOT_ABOVE)
// is needed even if it is non-conditional
.filter(l -> !l[4].isBlank() || l[0].equals("0130"))
.map(l -> new Entry(l[0],
Arrays.asList(l[1].trim().split(" ")),
Arrays.asList(l[3].trim().split(" ")),
l[4].replaceFirst("[A-Z].*$", "").trim(),
l[4].replaceFirst("^[ a-z]*", "").toUpperCase(Locale.ROOT).trim()))
.toList();
// Generate output file
Files.write(gensrcFile,
Files.lines(templateFile)
.flatMap(l -> {
if (l.trim().equals("%%%SpecialCasing%%%")) {
return entries.stream().map(GenerateSpecialCasing::entryToString);
} else {
return Stream.of(l);
}
})
.collect(Collectors.toList()),
StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING);
}
static String entryToString(Entry e) {
var codePoint = e.codePoint();
if (codePoint == null || codePoint.isBlank()) {
throw new RuntimeException("Corrupt entry: " + e);
}
return " new Entry(%s, new char[]{%s}, new char[]{%s}, %s, %s),"
.formatted(
"0x" + codePoint,
e.lowerCase().stream().map(cp -> cp.isEmpty() ? "" : "0x"+cp).collect(Collectors.joining(",")),
e.upperCase().stream().map(cp -> cp.isEmpty() ? "" : "0x"+cp).collect(Collectors.joining(",")),
"\"" + e.language() + "\"",
e.condition().isEmpty() ? "NONE" : e.condition());
}
}

83
make/langtools/src/classes/build/tools/symbolgenerator/CreateSymbols.java
View File

@ -183,6 +183,18 @@ import static java.lang.classfile.ClassFile.ACC_PUBLIC;
*/
public class CreateSymbols {
/**
* <p>Support for a "preview version" of classfiles when running with preview
* mode. This is modeled as a new version (@) and since preview mode is only
* supported for the current version, a single identifier token is sufficient.
*
* <p>For example, inside ct.sym, 27 will be modeled as 'R', and the preview
* for 27 will be '@'. Classfiles unchanged between 27 and 27-preview will
* not be duplicated (in the same way classfiles that are common between 26
* and 27 are shared).
*/
private static final String PREVIEW_VERSION = "@";
//<editor-fold defaultstate="collapsed" desc="ct.sym construction">
/**Create sig files for ct.sym reading the classes description from the directory that contains
* {@code ctDescriptionFile}, using the file as a recipe to create the sigfiles.
@ -212,12 +224,17 @@ public class CreateSymbols {
loadVersionClassesFromDirectory(data.classes, data.modules, moduleClassPath,
includedModules, currentVersion, previousVersion);
loadVersionClassesFromDirectory(data.classes, data.modules, moduleClassPath,
includedModules, PREVIEW_VERSION, currentVersion);
stripNonExistentAnnotations(data);
splitHeaders(data.classes);
Map<String, Map<Character, String>> package2Version2Module = new HashMap<>();
Map<String, Set<FileData>> directory2FileData = new TreeMap<>();
String currentVersionFin = currentVersion;
for (ModuleDescription md : data.modules.values()) {
for (ModuleHeaderDescription mhd : md.header) {
writeModulesForVersions(directory2FileData,
@ -226,6 +243,9 @@ public class CreateSymbols {
mhd.versions,
version -> {
String versionString = Character.toString(version);
if (PREVIEW_VERSION.equals(versionString)) {
versionString = currentVersionFin;
}
int versionNumber = Integer.parseInt(versionString, Character.MAX_RADIX);
versionString = Integer.toString(versionNumber);
if (versionNumber == currentVersionParsed && !preReleaseTag.isEmpty()) {
@ -809,6 +829,9 @@ public class CreateSymbols {
String module,
String version) throws IOException {
var classFile = ClassFile.of().build(ClassDesc.ofInternalName(classDescription.name), clb -> {
if (header.preview) {
clb.withVersion(ClassFile.latestMajorVersion(), ClassFile.PREVIEW_MINOR_VERSION);
}
if (header.extendsAttr != null)
clb.withSuperclass(ClassDesc.ofInternalName(header.extendsAttr));
clb.withInterfaceSymbols(header.implementsAttr.stream().map(ClassDesc::ofInternalName).collect(Collectors.toList()))
@ -1305,8 +1328,9 @@ public class CreateSymbols {
Collections.emptySet());
try {
record ExportedDir(Path modulePath, Path exportedDir) {}
Map<Path, ModuleHeaderDescription> modulePath2Header = new HashMap<>();
List<Path> pendingExportedDirectories = new ArrayList<>();
List<ExportedDir> pendingExportedDirectories = new ArrayList<>();
try (DirectoryStream<Path> ds = Files.newDirectoryStream(modulesDirectory)) {
for (Path p : ds) {
@ -1314,7 +1338,7 @@ public class CreateSymbols {
continue;
}
Path moduleInfo = p.resolve("module-info.class");
Path moduleInfo = resolvePossiblyPreviewClassFile(version, p, p.resolve("module-info.class"));
if (Files.isReadable(moduleInfo)) {
ModuleDescription md = inspectModuleInfoClassFile(Files.readAllBytes(moduleInfo),
@ -1333,7 +1357,7 @@ public class CreateSymbols {
for (String dir : currentModuleExports) {
includes.add(dir);
pendingExportedDirectories.add(p.resolve(dir));
pendingExportedDirectories.add(new ExportedDir(p, p.resolve(dir)));
}
} else {
throw new IllegalArgumentException("Included module: " +
@ -1345,13 +1369,15 @@ public class CreateSymbols {
List<String> pendingExtraClasses = new ArrayList<>();
for (Path exported : pendingExportedDirectories) {
try (DirectoryStream<Path> ds = Files.newDirectoryStream(exported)) {
for (ExportedDir exported : pendingExportedDirectories) {
try (DirectoryStream<Path> ds = Files.newDirectoryStream(exported.exportedDir())) {
for (Path p2 : ds) {
if (!Files.isRegularFile(p2) || !p2.getFileName().toString().endsWith(".class")) {
continue;
}
p2 = resolvePossiblyPreviewClassFile(version, exported.modulePath(), p2);
loadFromDirectoryHandleClassFile(p2, currentVersionClasses,
currentEIList, version,
pendingExtraClasses);
@ -1370,6 +1396,7 @@ public class CreateSymbols {
Path currentPath = e.getKey().resolve(current + ".class");
if (Files.isReadable(currentPath)) {
currentPath = resolvePossiblyPreviewClassFile(version, e.getKey(), currentPath);
String pack = current.substring(0, current.lastIndexOf('/'));
e.getValue().extraModulePackages.add(pack);
@ -1402,6 +1429,21 @@ public class CreateSymbols {
}
}
private Path resolvePossiblyPreviewClassFile(String version, Path moduleClassDir, Path classfile) {
if (!PREVIEW_VERSION.equals(version)) {
return classfile;
}
Path relativePath = moduleClassDir.relativize(classfile);
Path previewCandidate = moduleClassDir.resolve("META-INF").resolve("preview").resolve(relativePath);
if (Files.exists(previewCandidate)) {
return previewCandidate;
}
return classfile;
}
private void finishClassLoading(ClassList classes, Map<String, ModuleDescription> modules, Map<String, ModuleDescription> currentVersionModules, ClassList currentVersionClasses, ExcludeIncludeList currentEIList, String version,
String baseline) {
ModuleDescription unsupported =
@ -1930,6 +1972,7 @@ public class CreateSymbols {
ClassHeaderDescription headerDesc = new ClassHeaderDescription();
headerDesc.flags = cm.flags().flagsMask();
headerDesc.preview = cm.minorVersion() == ClassFile.PREVIEW_MINOR_VERSION;
if (cm.superclass().isPresent()) {
headerDesc.extendsAttr = cm.superclass().get().asInternalName();
@ -1996,6 +2039,7 @@ public class CreateSymbols {
headerDesc.versions = version;
headerDesc.flags = cm.flags().flagsMask();
headerDesc.preview = cm.minorVersion() == ClassFile.PREVIEW_MINOR_VERSION;
for (var attr : cm.attributes()) {
if (!readAttribute(headerDesc, attr))
@ -2263,7 +2307,13 @@ public class CreateSymbols {
feature.classTypeAnnotations = typeAnnotations2Descriptions(a.annotations());
case RuntimeVisibleTypeAnnotationsAttribute a ->
feature.runtimeTypeAnnotations = typeAnnotations2Descriptions(a.annotations());
default -> throw new IllegalArgumentException("Unhandled attribute: " + attr.attributeName()); // Do nothing
default -> {
if (attr.attributeName().equalsString("LoadableDescriptors")) {
//OK, do nothing
} else {
throw new IllegalArgumentException("Unhandled attribute: " + attr.attributeName());
}
}
}
return true;
@ -3308,11 +3358,13 @@ public class CreateSymbols {
static abstract class HeaderDescription extends FeatureDescription {
List<InnerClassInfo> innerClasses;
boolean preview;
@Override
public int hashCode() {
int hash = super.hashCode();
hash = 19 * hash + Objects.hashCode(this.innerClasses);
hash = 19 * hash + Objects.hashCode(this.preview);
return hash;
}
@ -3328,6 +3380,9 @@ public class CreateSymbols {
if (!listEquals(this.innerClasses, other.innerClasses)) {
return false;
}
if (this.preview != other.preview) {
return false;
}
return true;
}
@ -3366,6 +3421,22 @@ public class CreateSymbols {
}
}
@Override
protected void writeAttributes(Appendable output) throws IOException {
super.writeAttributes(output);
if (preview) {
output.append(" preview true");
}
}
@Override
protected void readAttributes(LineBasedReader reader) {
super.readAttributes(reader);
String inPreview = reader.attributes.get("preview");
if ("true".equals(inPreview)) {
preview = true;
}
}
}
static class MethodDescription extends FeatureDescription {

8
make/langtools/test/crules/DefinedByAnalyzer/Test.java
View File

@ -12,14 +12,10 @@ import com.sun.tools.javac.util.DefinedBy.Api;
public class Test implements SourcePositions, TaskListener {
@Override @DefinedBy(Api.COMPILER_TREE)
public long getStartPosition(CompilationUnitTree file, Tree tree) {
public long getStartPosition(Tree tree) {
return 0;
}
@Override
public long getEndPosition(CompilationUnitTree file, Tree tree) {
return 0;
}
@DefinedBy(Api.COMPILER_TREE)
@Override @DefinedBy(Api.COMPILER_TREE)
public long getEndPosition(Tree tree) {
return 0;
}

1
make/modules/java.base/Gensrc.gmk
View File

@ -37,6 +37,7 @@ include gensrc/GensrcMisc.gmk
include gensrc/GensrcModuleLoaderMap.gmk
include gensrc/GensrcRegex.gmk
include gensrc/GensrcScopedMemoryAccess.gmk
include gensrc/GensrcValueClasses.gmk
include gensrc/GensrcVarHandles.gmk
################################################################################

45
make/modules/java.base/gensrc/GensrcCharacterData.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -34,6 +34,9 @@ GENSRC_CHARACTERDATA :=
CHARACTERDATA_TEMPLATES = $(MODULE_SRC)/share/classes/java/lang
UNICODEDATA = $(MODULE_SRC)/share/data/unicodedata
SPECIALCASING = $(UNICODEDATA)/SpecialCasing.txt
DERIVEDCOREPROPS = $(UNICODEDATA)/DerivedCoreProperties.txt
PROPLIST = $(UNICODEDATA)/PropList.txt
ifneq ($(DEBUG_LEVEL), release)
ifeq ($(ALLOW_ABSOLUTE_PATHS_IN_OUTPUT), true)
@ -49,9 +52,9 @@ define SetupCharacterData
$(TOOL_GENERATECHARACTER) $2 $(DEBUG_OPTION) \
-template $(CHARACTERDATA_TEMPLATES)/$1.java.template \
-spec $(UNICODEDATA)/UnicodeData.txt \
-specialcasing $(UNICODEDATA)/SpecialCasing.txt \
-proplist $(UNICODEDATA)/PropList.txt \
-derivedprops $(UNICODEDATA)/DerivedCoreProperties.txt \
-specialcasing $(SPECIALCASING) \
-proplist $(PROPLIST) \
-derivedprops $(DERIVEDCOREPROPS) \
-emojidata $(UNICODEDATA)/emoji/emoji-data.txt \
-o $(SUPPORT_OUTPUTDIR)/gensrc/java.base/java/lang/$1.java \
-usecharforbyte $3
@ -88,6 +91,40 @@ $(GENSRC_STRINGCASEFOLDING): $(BUILD_TOOLS_JDK) $(STRINGCASEFOLDING_TEMPLATE) $(
TARGETS += $(GENSRC_STRINGCASEFOLDING)
################################################################################
# Rule to create $(SUPPORT_OUTPUTDIR)/gensrc/java.base/java/lang/ConditionalSpecialCasing.java
################################################################################
GENSRC_CONDITIONALSPECIALCASING := $(SUPPORT_OUTPUTDIR)/gensrc/java.base/java/lang/ConditionalSpecialCasing.java
GENSRC_CONDITIONALSPECIALCASING_SPECIALCASING_TMP := $(GENSRC_CONDITIONALSPECIALCASING).specialcasing.tmp
GENSRC_CONDITIONALSPECIALCASING_PROPS_TMP := $(GENSRC_CONDITIONALSPECIALCASING).props.tmp
CONDITIONALSPECIALCASINGTEMP := $(MODULE_SRC)/share/classes/java/lang/ConditionalSpecialCasing.java.template
FINALSIGMAPARAMS := Cased Case_Ignorable
SOFTDOTTEDPARAMS := Soft_Dotted
$(GENSRC_CONDITIONALSPECIALCASING): $(BUILD_TOOLS_JDK) $(CONDITIONALSPECIALCASINGTEMP) \
$(SPECIALCASING) $(DERIVEDCOREPROPS) $(PROPLIST)
$(call LogInfo, Generating $@)
$(call MakeTargetDir)
$(TOOL_GENERATESPECIALCASING) \
$(CONDITIONALSPECIALCASINGTEMP) \
$(SPECIALCASING) \
$(GENSRC_CONDITIONALSPECIALCASING_SPECIALCASING_TMP)
$(TOOL_GENERATEEXTRAPROPERTIES) \
$(GENSRC_CONDITIONALSPECIALCASING_SPECIALCASING_TMP) \
$(DERIVEDCOREPROPS) \
$(GENSRC_CONDITIONALSPECIALCASING_PROPS_TMP) \
$(FINALSIGMAPARAMS)
$(TOOL_GENERATEEXTRAPROPERTIES) \
$(GENSRC_CONDITIONALSPECIALCASING_PROPS_TMP) \
$(PROPLIST) \
$(GENSRC_CONDITIONALSPECIALCASING) \
$(SOFTDOTTEDPARAMS)
$(RM) $(GENSRC_CONDITIONALSPECIALCASING_SPECIALCASING_TMP)
$(RM) $(GENSRC_CONDITIONALSPECIALCASING_PROPS_TMP)
TARGETS += $(GENSRC_CONDITIONALSPECIALCASING)
endif # include guard
include MakeIncludeEnd.gmk

76
make/modules/java.base/gensrc/GensrcValueClasses.gmk Normal file
View File

@ -0,0 +1,76 @@
#
# Copyright (c) 2023, 2024, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 only, as
# published by the Free Software Foundation. Oracle designates this
# particular file as subject to the "Classpath" exception as provided
# by Oracle in the LICENSE file that accompanied this code.
#
# This code is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# version 2 for more details (a copy is included in the LICENSE file that
# accompanied this code).
#
# You should have received a copy of the GNU General Public License version
# 2 along with this work; if not, write to the Free Software Foundation,
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
#
# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
# or visit www.oracle.com if you need additional information or have any
# questions.
#
################################################################################
# Generate the value class replacements for selected java.base source files
# See doc/value-class-preview.md for an overview of value class generation
JAVA_BASE_VALUE_CLASS_REPLACEMENTS := \
java/lang/Byte.java \
java/lang/Short.java \
java/lang/Integer.java \
java/lang/Long.java \
java/lang/Float.java \
java/lang/Double.java \
java/lang/Boolean.java \
java/lang/Character.java \
java/lang/Number.java \
java/lang/Record.java \
java/util/Optional.java \
java/util/OptionalInt.java \
java/util/OptionalLong.java \
java/util/OptionalDouble.java \
java/time/LocalDate.java \
java/time/LocalDateTime.java \
java/time/LocalTime.java \
java/time/Duration.java \
java/time/Instant.java \
java/time/MonthDay.java \
java/time/ZonedDateTime.java \
java/time/OffsetDateTime.java \
java/time/OffsetTime.java \
java/time/YearMonth.java \
java/time/Year.java \
java/time/Period.java \
java/time/chrono/ChronoLocalDateImpl.java \
java/time/chrono/MinguoDate.java \
java/time/chrono/HijrahDate.java \
java/time/chrono/JapaneseDate.java \
java/time/chrono/ThaiBuddhistDate.java \
#
JAVA_BASE_VALUE_CLASS_SRC_PATHS := \
$(foreach f, $(JAVA_BASE_VALUE_CLASS_REPLACEMENTS), $(addprefix $(TOPDIR)/src/java.base/share/classes/, $(f)))
$(eval $(call SetupTextFileProcessing, JAVA_BASE_VALUE_CLASS_TARGETS, \
SOURCE_FILES := $(JAVA_BASE_VALUE_CLASS_SRC_PATHS), \
SOURCE_BASE_DIR := $(TOPDIR)/src/java.base/share/classes, \
OUTPUT_DIR := $(SUPPORT_OUTPUTDIR)/gensrc-valueclasses/java.base/, \
REPLACEMENTS := \
/\*value\*/ class => value class ; \
/\*value\*/ record => value record ; \
))
TARGETS += $(JAVA_BASE_VALUE_CLASS_TARGETS)

42
make/modules/java.base/gensrc/GensrcVarHandles.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2015, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2015, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -30,35 +30,59 @@ ifeq ($(INCLUDE), true)
VARHANDLES_INPUT_DIR := $(MODULE_SRC)/share/classes/java/lang/invoke
VARHANDLES_OUTPUT_DIR := $(SUPPORT_OUTPUTDIR)/gensrc/java.base/java/lang/invoke
# These guys are cased because make cannot preserve case...
VARHANDLE_OBJECT_TYPES := Reference FlatValue NonAtomicReference NonAtomicFlatValue
################################################################################
# Setup a rule for generating a VarHandle java class
#
# arg $1: type for this varhandle
define GenerateVarHandle
VARHANDLE_$1_type := $$(strip $$(if $$(filter reference, $1), Object, $1))
VARHANDLE_$1_Type := $$(call Conv, $1, Type)
# Underlying erased signature type, Object or a primitive type
VARHANDLE_$1_type := \
$$(strip $$(if $$(filter $(VARHANDLE_OBJECT_TYPES), $1), Object, $1))
VARHANDLE_$1_InputType := \
$$(strip $$(if $$(filter $(VARHANDLE_OBJECT_TYPES), $1), $1, \
$$(call Conv, $1, Type)))
VARHANDLE_$1_Type := \
$$(strip $$(subst NonAtomicReference, Reference, \
$$(subst NonAtomicFlatValue, FlatValue, $$(VARHANDLE_$1_InputType))))
$1_KEYS := $$(VARHANDLE_$1_type) CAS
ifneq ($$(filter $(PRIMITIVE_TYPES), $1),)
# Reference types use ArrayVarHandle class
$1_KEYS += Array Static
else ifneq ($$(filter Reference NonAtomicReference, $1),)
$1_KEYS += Reference Static
else ifneq ($$(filter FlatValue NonAtomicFlatValue, $1),)
# No static field is flat in Hotspot
$1_KEYS += FlatValue
endif
ifneq ($$(filter byte short char, $1),)
$1_KEYS += ShorterThanInt
endif
ifeq ($$(filter boolean reference, $1),)
ifneq ($$(filter $(NUMBER_TYPES), $1),)
$1_KEYS += AtomicAdd
endif
ifeq ($$(filter float double reference, $1),)
ifneq ($$(filter $(BITWISE_PRIMITIVE_TYPES), $1),)
$1_KEYS += Bitwise
endif
ifeq ($$(filter NonAtomicReference NonAtomicFlatValue, $1),)
# Everyone except NonAtomicXxx have non-plain access
$1_KEYS += NonPlainAccess
endif
$$(eval $$(call SetupStreamPreProcessing, GEN_VARHANDLE_$1, \
SOURCE_FILE := $$(VARHANDLES_INPUT_DIR)/X-VarHandle.java.template, \
OUTPUT_FILE := $$(VARHANDLES_OUTPUT_DIR)/VarHandle$$(VARHANDLE_$1_Type)s.java, \
OUTPUT_FILE := $$(VARHANDLES_OUTPUT_DIR)/VarHandle$$(VARHANDLE_$1_InputType)s.java, \
INFO := Generating VarHandle class for $1, \
SUBST_EMPTY_LINES := false, \
KEYS := $$($1_KEYS), \
REPLACEMENTS := \
type=$$(VARHANDLE_$1_type) \
Type=$$(VARHANDLE_$1_Type), \
Type=$$(VARHANDLE_$1_Type) \
InputType=$$(VARHANDLE_$1_InputType), \
))
TARGETS += $$(GEN_VARHANDLE_$1)
endef
@ -111,7 +135,7 @@ define GenerateVarHandleMemorySegment
$1_KEYS += CAS
endif
ifneq ($$(filter boolean byte, $1),)
$1_KEYS += byte
$1_KEYS += ByteOrBoolean
endif
ifneq ($$(filter float double, $1),)
$1_KEYS += floatingPoint
@ -143,7 +167,7 @@ endef
################################################################################
# Generate all VarHandle related classes
$(foreach t, $(PRIMITIVE_TYPES) reference, \
$(foreach t, $(PRIMITIVE_TYPES) $(VARHANDLE_OBJECT_TYPES), \
$(eval $(call GenerateVarHandle,$t)) \
)

2
make/modules/java.base/lib/CoreLibraries.gmk
View File

@ -57,8 +57,6 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBJAVA, \
ProcessImpl_md.c_CFLAGS := $(VERSION_CFLAGS), \
java_props_md.c_CFLAGS := \
-DARCHPROPNAME='"$(OPENJDK_TARGET_CPU_OSARCH)"', \
DISABLED_WARNINGS_gcc_ProcessImpl_md.c := unused-result, \
DISABLED_WARNINGS_clang_TimeZone_md.c := unused-variable, \
JDK_LIBS := libjvm, \
LIBS_linux := $(LIBDL), \
LIBS_aix := $(LIBDL) $(LIBM), \

84
make/modules/java.desktop/Lib.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -37,49 +37,51 @@ $(call FillFindCache, $(wildcard $(TOPDIR)/src/java.desktop/*/native))
include lib/AwtLibraries.gmk
include lib/ClientLibraries.gmk
ifeq ($(call isTargetOs, aix), false)
##############################################################################
# Build libjsound
##############################################################################
ifeq ($(ENABLE_JSOUND), true)
ifeq ($(call isTargetOs, aix), false)
##############################################################################
# Build libjsound
##############################################################################
LIBJSOUND_CFLAGS := \
$(ALSA_CFLAGS) \
-DX_PLATFORM=X_$(OPENJDK_TARGET_OS_UPPERCASE) \
-DUSE_PORTS=TRUE \
-DUSE_DAUDIO=TRUE \
-DUSE_PLATFORM_MIDI_OUT=TRUE \
-DUSE_PLATFORM_MIDI_IN=TRUE \
#
LIBJSOUND_CFLAGS := \
$(ALSA_CFLAGS) \
-DX_PLATFORM=X_$(OPENJDK_TARGET_OS_UPPERCASE) \
-DUSE_PORTS=TRUE \
-DUSE_DAUDIO=TRUE \
-DUSE_PLATFORM_MIDI_OUT=TRUE \
-DUSE_PLATFORM_MIDI_IN=TRUE \
#
LIBJSOUND_LINK_TYPE := C
ifeq ($(call isTargetOs, macosx), true)
LIBJSOUND_LINK_TYPE := C++
LIBJSOUND_LINK_TYPE := C
ifeq ($(call isTargetOs, macosx), true)
LIBJSOUND_LINK_TYPE := C++
endif
$(eval $(call SetupJdkLibrary, BUILD_LIBJSOUND, \
NAME := jsound, \
LINK_TYPE := $(LIBJSOUND_LINK_TYPE), \
OPTIMIZATION := LOW, \
EXTRA_HEADER_DIRS := java.base:libjava, \
CFLAGS := $(LIBJSOUND_CFLAGS), \
CXXFLAGS := $(LIBJSOUND_CFLAGS), \
DISABLED_WARNINGS_gcc := undef unused-variable, \
DISABLED_WARNINGS_clang := undef unused-variable, \
DISABLED_WARNINGS_clang_PLATFORM_API_MacOSX_MidiUtils.c := \
unused-but-set-variable, \
DISABLED_WARNINGS_clang_DirectAudioDevice.c := unused-function, \
LIBS_linux := $(ALSA_LIBS), \
LIBS_macosx := \
-framework AudioToolbox \
-framework AudioUnit \
-framework CoreAudio \
-framework CoreFoundation \
-framework CoreMIDI \
-framework CoreServices, \
LIBS_windows := advapi32.lib dsound.lib ole32.lib user32.lib winmm.lib, \
))
TARGETS += $(BUILD_LIBJSOUND)
endif
$(eval $(call SetupJdkLibrary, BUILD_LIBJSOUND, \
NAME := jsound, \
LINK_TYPE := $(LIBJSOUND_LINK_TYPE), \
OPTIMIZATION := LOW, \
EXTRA_HEADER_DIRS := java.base:libjava, \
CFLAGS := $(LIBJSOUND_CFLAGS), \
CXXFLAGS := $(LIBJSOUND_CFLAGS), \
DISABLED_WARNINGS_gcc := undef unused-variable, \
DISABLED_WARNINGS_clang := undef unused-variable, \
DISABLED_WARNINGS_clang_PLATFORM_API_MacOSX_MidiUtils.c := \
unused-but-set-variable, \
DISABLED_WARNINGS_clang_DirectAudioDevice.c := unused-function, \
LIBS_linux := $(ALSA_LIBS), \
LIBS_macosx := \
-framework AudioToolbox \
-framework AudioUnit \
-framework CoreAudio \
-framework CoreFoundation \
-framework CoreMIDI \
-framework CoreServices, \
LIBS_windows := advapi32.lib dsound.lib ole32.lib user32.lib winmm.lib, \
))
TARGETS += $(BUILD_LIBJSOUND)
endif
ifeq ($(call isTargetOs, macosx), true)

4
make/modules/java.management/Lib.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -36,7 +36,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBMANAGEMENT, \
OPTIMIZATION := HIGH, \
JDK_LIBS := java.base:libjava java.base:libjvm, \
LIBS_aix := -lperfstat, \
LIBS_windows := advapi32.lib psapi.lib, \
LIBS_windows := advapi32.lib, \
))
TARGETS += $(BUILD_LIBMANAGEMENT)

12
make/modules/jdk.attach/Lib.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -31,20 +31,12 @@ include LibCommon.gmk
## Build libattach
################################################################################
ifeq ($(call isTargetOs, windows), true)
# In (at least) VS2013 and later, -DPSAPI_VERSION=1 is needed to generate
# a binary that is compatible with windows versions older than 7/2008R2.
# See MSDN documentation for GetProcessMemoryInfo for more information.
LIBATTACH_CFLAGS := -DPSAPI_VERSION=1
endif
$(eval $(call SetupJdkLibrary, BUILD_LIBATTACH, \
NAME := attach, \
OPTIMIZATION := LOW, \
CFLAGS := $(LIBATTACH_CFLAGS), \
CFLAGS_windows := -Gy, \
JDK_LIBS := java.base:libjava, \
LIBS_windows := advapi32.lib psapi.lib, \
LIBS_windows := advapi32.lib, \
))
TARGETS += $(BUILD_LIBATTACH)

3
make/modules/jdk.hotspot.agent/Lib.gmk
View File

@ -60,6 +60,9 @@ LIBSAPROC_EXCLUDE_FILES :=
ifneq ($(call And, $(call isTargetOs, linux) $(call isTargetCpu, x86_64 aarch64)), true)
LIBSAPROC_EXCLUDE_FILES := DwarfParser.cpp dwarf.cpp
endif
ifneq ($(call And, $(call isTargetOs, linux) $(call isTargetCpu, aarch64)), true)
LIBSAPROC_EXCLUDE_FILES += AARCH64DwarfParser.cpp aarch64Dwarf.cpp LinuxAARCH64DebuggerLocal.cpp
endif
$(eval $(call SetupJdkLibrary, BUILD_LIBSAPROC, \
NAME := saproc, \

14
make/modules/jdk.management/Lib.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2015, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2015, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -31,23 +31,13 @@ include LibCommon.gmk
## Build libmanagement_ext
################################################################################
ifeq ($(call isTargetOs, windows), true)
# In (at least) VS2013 and later, -DPSAPI_VERSION=1 is needed to generate
# a binary that is compatible with windows versions older than 7/2008R2.
# See MSDN documentation for GetProcessMemoryInfo for more information.
LIBMANAGEMENT_EXT_CFLAGS := -DPSAPI_VERSION=1
endif
$(eval $(call SetupJdkLibrary, BUILD_LIBMANAGEMENT_EXT, \
NAME := management_ext, \
OPTIMIZATION := HIGH, \
DISABLED_WARNINGS_gcc_DiagnosticCommandImpl.c := unused-variable, \
DISABLED_WARNINGS_clang_DiagnosticCommandImpl.c := unused-variable, \
DISABLED_WARNINGS_clang_UnixOperatingSystem.c := format-nonliteral, \
CFLAGS := $(LIBMANAGEMENT_EXT_CFLAGS), \
JDK_LIBS := java.base:libjava java.base:libjvm, \
LIBS_aix := -lperfstat, \
LIBS_windows := advapi32.lib psapi.lib, \
LIBS_windows := advapi32.lib, \
))
TARGETS += $(BUILD_LIBMANAGEMENT_EXT)

88
make/test/BuildJtregValueClassPlugin.gmk Normal file
View File

@ -0,0 +1,88 @@
#
# Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 only, as
# published by the Free Software Foundation. Oracle designates this
# particular file as subject to the "Classpath" exception as provided
# by Oracle in the LICENSE file that accompanied this code.
#
# This code is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# version 2 for more details (a copy is included in the LICENSE file that
# accompanied this code).
#
# You should have received a copy of the GNU General Public License version
# 2 along with this work; if not, write to the Free Software Foundation,
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
#
# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
# or visit www.oracle.com if you need additional information or have any
# questions.
#
include MakeFileStart.gmk
################################################################################
# Builds one JAR used by jtreg to test value classes (JEP 401):
#
# valueClassPlugin.jar -- Contains ValueClassPlugin (a javac Plugin that
# rewrites @AsValueClass classes to value classes at parse time) together
# with @AsValueClass and the META-INF service descriptor, compiled WITH
# --enable-preview and the required internal-API exports.
#
# Usage in test runs (via RunTests.gmk):
# make test TEST=... JTREG=VALUE_CLASS_PLUGIN=true
#
# The plugin is also enabled automatically when --enable-preview is passed via
# VM_OPTIONS or JAVA_OPTIONS and the plugin JARs are present in the test image:
# make test TEST=... JTREG=VM_OPTIONS=--enable-preview
#
################################################################################
include CopyFiles.gmk
include JavaCompilation.gmk
VCP_BASEDIR := $(TOPDIR)/test/jtreg_value_class_plugin
VCP_SUPPORT := $(SUPPORT_OUTPUTDIR)/test/jtreg_value_class_plugin
# Compile ValueClassPlugin WITH --enable-preview
$(eval $(call SetupJavaCompilation, BUILD_VCP_PLUGIN, \
TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \
SRC := $(VCP_BASEDIR)/plugin, \
BIN := $(VCP_SUPPORT)/plugin_classes, \
JAR := $(VCP_SUPPORT)/valueClassPlugin.jar, \
JAVAC_FLAGS := \
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
--enable-preview, \
DISABLED_WARNINGS := preview, \
))
TARGETS += $(BUILD_VCP_PLUGIN)
################################################################################
# Targets for building test-image.
################################################################################
$(eval $(call SetupCopyFiles, COPY_VCP, \
SRC := $(VCP_SUPPORT), \
DEST := $(TEST_IMAGE_DIR)/jtreg_value_class_plugin, \
FILES := \
$(VCP_SUPPORT)/valueClassPlugin.jar, \
))
IMAGES_TARGETS += $(COPY_VCP)
build: $(TARGETS)
images: $(IMAGES_TARGETS)
.PHONY: images
################################################################################
include MakeFileEnd.gmk

8
make/test/BuildMicrobenchmark.gmk
View File

@ -78,13 +78,14 @@ MICROBENCHMARK_MANIFEST := Build: $(FULL_VERSION)\n\
# requires the use of -processor option during benchmark compilation.
# Build microbenchmark suite for the current JDK
# Need to patch java.base to include preview classes not found in interim javac
$(eval $(call SetupJavaCompilation, BUILD_JDK_MICROBENCHMARK, \
TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \
SMALL_JAVA := false, \
CLASSPATH := $(JMH_COMPILE_JARS), \
CREATE_API_DIGEST := true, \
DISABLED_WARNINGS := restricted this-escape rawtypes removal cast \
serial preview, \
DISABLED_WARNINGS := restricted this-escape processing rawtypes removal cast \
serial preview unchecked deprecation dangling-doc-comments, \
SRC := $(MICROBENCHMARK_SRC), \
BIN := $(MICROBENCHMARK_CLASSES), \
JAVAC_FLAGS := \
@ -95,12 +96,15 @@ $(eval $(call SetupJavaCompilation, BUILD_JDK_MICROBENCHMARK, \
--add-exports java.base/jdk.internal.jimage=ALL-UNNAMED \
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED \
--add-exports java.base/jdk.internal.util=ALL-UNNAMED \
--add-exports java.base/jdk.internal.value=ALL-UNNAMED \
--add-exports java.base/jdk.internal.vm=ALL-UNNAMED \
--add-exports java.base/jdk.internal.vm.annotation=ALL-UNNAMED \
--add-exports java.base/sun.invoke.util=ALL-UNNAMED \
--add-exports java.base/sun.security.util=ALL-UNNAMED \
--add-exports java.base/sun.security.util.math=ALL-UNNAMED \
--add-exports java.base/sun.security.util.math.intpoly=ALL-UNNAMED \
--enable-preview \
--patch-module java.base=$(SUPPORT_OUTPUTDIR)/preview/java.base \
-XDsuppressNotes \
-processor org.openjdk.jmh.generators.BenchmarkProcessor \
-s $(MICROBENCHMARK_GENSRC), \

10
make/test/BuildTestLib.gmk
View File

@ -46,6 +46,7 @@ $(eval $(call SetupJavaCompilation, BUILD_WB_JAR, \
SRC := $(TEST_LIB_SOURCE_DIR)/jdk/test/whitebox/, \
BIN := $(TEST_LIB_SUPPORT)/wb_classes, \
JAR := $(TEST_LIB_SUPPORT)/wb.jar, \
DISABLED_WARNINGS := preview, \
JAVAC_FLAGS := --enable-preview, \
))
@ -58,10 +59,11 @@ endif
$(eval $(call SetupJavaCompilation, BUILD_TEST_LIB_JAR, \
TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \
SRC := $(TEST_LIB_SOURCE_DIR), \
EXCLUDES := $(BUILD_TEST_LIB_JAR_EXCLUDES), \
EXCLUDES := $(BUILD_TEST_LIB_JAR_EXCLUDES) org, \
BIN := $(TEST_LIB_SUPPORT)/test-lib_classes, \
HEADERS := $(TEST_LIB_SUPPORT)/test-lib_headers, \
JAR := $(TEST_LIB_SUPPORT)/test-lib.jar, \
DISABLED_WARNINGS := preview, \
JAVAC_FLAGS := --add-exports java.base/sun.security.util=ALL-UNNAMED \
--add-exports java.base/jdk.internal.classfile=ALL-UNNAMED \
--add-exports java.base/jdk.internal.classfile.attribute=ALL-UNNAMED \
@ -72,6 +74,12 @@ $(eval $(call SetupJavaCompilation, BUILD_TEST_LIB_JAR, \
--add-exports java.base/sun.security.provider.certpath=ALL-UNNAMED \
--add-exports java.base/sun.security.tools.keytool=ALL-UNNAMED \
--add-exports java.base/sun.security.x509=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
--enable-preview, \
))

264
src/hotspot/cpu/aarch64/aarch64.ad
View File

@ -1198,8 +1198,12 @@ class HandlerImpl {
static int emit_deopt_handler(C2_MacroAssembler* masm);
static uint size_deopt_handler() {
// count one branch instruction and one far call instruction sequence
return NativeInstruction::instruction_size + MacroAssembler::far_codestub_branch_size();
bool use_far_branch = MacroAssembler::target_needs_far_branch(SharedRuntime::deopt_blob()->unpack());
// far: adrp, add, blr; near: bl
uint target_branch_instructions = use_far_branch ? 3 : 1;
// target branch + one branch instruction
uint deopt_handler_instructions = target_branch_instructions + 1;
return deopt_handler_instructions * NativeInstruction::instruction_size;
}
};
@ -1658,19 +1662,19 @@ bool needs_acquiring_load_exclusive(const Node *n)
// from the start of the call to the point where the return address
// will point.
int MachCallStaticJavaNode::ret_addr_offset()
int MachCallStaticJavaNode::ret_addr_offset() const
{
// call should be a simple bl
int off = 4;
return off;
}
int MachCallDynamicJavaNode::ret_addr_offset()
int MachCallDynamicJavaNode::ret_addr_offset() const
{
return 16; // movz, movk, movk, bl
}
int MachCallRuntimeNode::ret_addr_offset() {
int MachCallRuntimeNode::ret_addr_offset() const {
// for generated stubs the call will be
// bl(addr)
// or with far branches
@ -1684,6 +1688,9 @@ int MachCallRuntimeNode::ret_addr_offset() {
CodeBlob *cb = CodeCache::find_blob(_entry_point);
if (cb) {
return 1 * NativeInstruction::instruction_size;
} else if (_entry_point == nullptr) {
// See CallLeafNoFPIndirect
return 1 * NativeInstruction::instruction_size;
} else {
return 6 * NativeInstruction::instruction_size;
}
@ -1792,49 +1799,15 @@ void MachPrologNode::format(PhaseRegAlloc *ra_, outputStream *st) const {
void MachPrologNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
Compile* C = ra_->C;
// n.b. frame size includes space for return pc and rfp
const int framesize = C->output()->frame_size_in_bytes();
if (C->clinit_barrier_on_entry()) {
assert(!C->method()->holder()->is_not_initialized(), "initialization should have been started");
Label L_skip_barrier;
__ mov_metadata(rscratch2, C->method()->holder()->constant_encoding());
__ clinit_barrier(rscratch2, rscratch1, &L_skip_barrier);
__ far_jump(RuntimeAddress(SharedRuntime::get_handle_wrong_method_stub()));
__ bind(L_skip_barrier);
}
if (C->max_vector_size() > 0) {
__ reinitialize_ptrue();
}
int bangsize = C->output()->bang_size_in_bytes();
if (C->output()->need_stack_bang(bangsize))
__ generate_stack_overflow_check(bangsize);
__ build_frame(framesize);
__ verified_entry(C, 0);
if (C->stub_function() == nullptr) {
BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler();
// Dummy labels for just measuring the code size
Label dummy_slow_path;
Label dummy_continuation;
Label dummy_guard;
Label* slow_path = &dummy_slow_path;
Label* continuation = &dummy_continuation;
Label* guard = &dummy_guard;
if (!Compile::current()->output()->in_scratch_emit_size()) {
// Use real labels from actual stub when not emitting code for the purpose of measuring its size
C2EntryBarrierStub* stub = new (Compile::current()->comp_arena()) C2EntryBarrierStub();
Compile::current()->output()->add_stub(stub);
slow_path = &stub->entry();
continuation = &stub->continuation();
guard = &stub->guard();
}
// In the C2 code, we move the non-hot part of nmethod entry barriers out-of-line to a stub.
bs->nmethod_entry_barrier(masm, slow_path, continuation, guard);
__ entry_barrier();
}
if (!Compile::current()->output()->in_scratch_emit_size()) {
__ bind(*_verified_entry);
}
if (VerifyStackAtCalls) {
@ -1851,12 +1824,6 @@ void MachPrologNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
}
}
uint MachPrologNode::size(PhaseRegAlloc* ra_) const
{
return MachNode::size(ra_); // too many variables; just compute it
// the hard way
}
int MachPrologNode::reloc() const
{
return 0;
@ -1899,7 +1866,7 @@ void MachEpilogNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
Compile* C = ra_->C;
int framesize = C->output()->frame_slots() << LogBytesPerInt;
__ remove_frame(framesize);
__ remove_frame(framesize, C->needs_stack_repair());
if (StackReservedPages > 0 && C->has_reserved_stack_access()) {
__ reserved_stack_check();
@ -1918,11 +1885,6 @@ void MachEpilogNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
}
}
uint MachEpilogNode::size(PhaseRegAlloc *ra_) const {
// Variable size. Determine dynamically.
return MachNode::size(ra_);
}
int MachEpilogNode::reloc() const {
// Return number of relocatable values contained in this instruction.
return 1; // 1 for polling page.
@ -2221,8 +2183,47 @@ uint BoxLockNode::size(PhaseRegAlloc *ra_) const {
}
}
//=============================================================================
///=============================================================================
#ifndef PRODUCT
void MachVEPNode::format(PhaseRegAlloc* ra_, outputStream* st) const
{
st->print_cr("# MachVEPNode");
if (!_verified) {
st->print_cr("\t load_class");
} else {
st->print_cr("\t unpack_inline_arg");
}
}
#endif
void MachVEPNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc* ra_) const
{
if (!_verified) {
__ ic_check(1);
} else {
if (ra_->C->stub_function() == nullptr) {
// Emit the entry barrier in a temporary frame before unpacking because
// it can deopt, which would require packing the scalarized args again.
__ verified_entry(ra_->C, 0);
__ entry_barrier();
int framesize = ra_->C->output()->frame_slots() << LogBytesPerInt;
__ remove_frame(framesize, false);
}
// Unpack inline type args passed as oop and then jump to
// the verified entry point (skipping the unverified entry).
int sp_inc = __ unpack_inline_args(ra_->C, _receiver_only);
// Emit code for verified entry and save increment for stack repair on return
__ verified_entry(ra_->C, sp_inc);
if (Compile::current()->output()->in_scratch_emit_size()) {
Label dummy_verified_entry;
__ b(dummy_verified_entry);
} else {
__ b(*_verified_entry);
}
}
}
//=============================================================================
#ifndef PRODUCT
void MachUEPNode::format(PhaseRegAlloc* ra_, outputStream* st) const
{
@ -2239,11 +2240,6 @@ void MachUEPNode::emit(C2_MacroAssembler* masm, PhaseRegAlloc* ra_) const
__ ic_check(InteriorEntryAlignment);
}
uint MachUEPNode::size(PhaseRegAlloc* ra_) const
{
return MachNode::size(ra_);
}
// REQUIRED EMIT CODE
//=============================================================================
@ -2512,25 +2508,25 @@ uint Matcher::float_pressure_limit()
return (FLOATPRESSURE == -1) ? _FLOAT_REG_mask.size() : FLOATPRESSURE;
}
const RegMask& Matcher::divI_proj_mask() {
const RegMask& Matcher::firstI_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for MODI projection of divmodI.
const RegMask& Matcher::modI_proj_mask() {
// Register for the second projection of an int pair
const RegMask& Matcher::secondI_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for DIVL projection of divmodL.
const RegMask& Matcher::divL_proj_mask() {
// Register for the first projection of a long pair
const RegMask& Matcher::firstL_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for MODL projection of divmodL.
const RegMask& Matcher::modL_proj_mask() {
// Register for the second projection of a long pair
const RegMask& Matcher::secondL_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
@ -3687,6 +3683,37 @@ encode %{
// Check that stack depth is unchanged: find majik cookie on stack
__ call_Unimplemented();
}
if (tf()->returns_inline_type_as_fields() && !_method->is_method_handle_intrinsic() && _method->return_type()->is_loaded()) {
// The last return value is not set by the callee but used to pass the null marker to compiled code.
// Search for the corresponding projection, get the register and emit code that initializes it.
uint con = (tf()->range_cc()->cnt() - 1);
for (DUIterator_Fast imax, i = fast_outs(imax); i < imax; i++) {
ProjNode* proj = fast_out(i)->as_Proj();
if (proj->_con == con) {
// Set null marker if r0 is non-null (a non-null value is returned buffered or scalarized)
OptoReg::Name optoReg = ra_->get_reg_first(proj);
VMReg reg = OptoReg::as_VMReg(optoReg, ra_->_framesize, OptoReg::reg2stack(ra_->_matcher._new_SP));
Register toReg = reg->is_reg() ? reg->as_Register() : rscratch1;
__ cmp(r0, zr);
__ cset(toReg, Assembler::NE);
if (reg->is_stack()) {
int st_off = reg->reg2stack() * VMRegImpl::stack_slot_size;
__ str(toReg, Address(sp, st_off));
}
break;
}
}
if (return_value_is_used()) {
// An inline type is returned as fields in multiple registers.
// R0 either contains an oop if the inline type is buffered or a pointer
// to the corresponding InlineKlass with the lowest bit set to 1. Zero r0
// if the lowest bit is set to allow C2 to use the oop after null checking.
// r0 &= (r0 & 1) - 1
__ andr(rscratch1, r0, 0x1);
__ sub(rscratch1, rscratch1, 0x1);
__ andr(r0, r0, rscratch1);
}
}
%}
enc_class aarch64_enc_java_to_runtime(method meth) %{
@ -3976,6 +4003,16 @@ operand immI_le_4()
interface(CONST_INTER);
%}
operand immI_4()
%{
predicate(n->get_int() == 4);
match(ConI);
op_cost(0);
format %{ %}
interface(CONST_INTER);
%}
operand immI_16()
%{
predicate(n->get_int() == 16);
@ -8112,6 +8149,36 @@ instruct castX2P(iRegPNoSp dst, iRegL src) %{
ins_pipe(ialu_reg);
%}
instruct castI2N(iRegNNoSp dst, iRegI src) %{
match(Set dst (CastI2N src));
ins_cost(INSN_COST);
format %{ "mov $dst, $src\t# int -> narrow ptr" %}
ins_encode %{
if ($dst$$reg != $src$$reg) {
__ mov(as_Register($dst$$reg), as_Register($src$$reg));
}
%}
ins_pipe(ialu_reg);
%}
instruct castN2X(iRegLNoSp dst, iRegN src) %{
match(Set dst (CastP2X src));
ins_cost(INSN_COST);
format %{ "mov $dst, $src\t# ptr -> long" %}
ins_encode %{
if ($dst$$reg != $src$$reg) {
__ mov(as_Register($dst$$reg), as_Register($src$$reg));
}
%}
ins_pipe(ialu_reg);
%}
instruct castP2X(iRegLNoSp dst, iRegP src) %{
match(Set dst (CastP2X src));
@ -8224,7 +8291,7 @@ instruct encodeKlass_not_null(iRegNNoSp dst, iRegP src) %{
ins_encode %{
Register src_reg = as_Register($src$$reg);
Register dst_reg = as_Register($dst$$reg);
__ encode_klass_not_null(dst_reg, src_reg);
__ encode_klass_not_null(dst_reg, src_reg, rscratch1);
%}
ins_pipe(ialu_reg);
@ -8239,11 +8306,7 @@ instruct decodeKlass_not_null(iRegPNoSp dst, iRegN src) %{
ins_encode %{
Register src_reg = as_Register($src$$reg);
Register dst_reg = as_Register($dst$$reg);
if (dst_reg != src_reg) {
__ decode_klass_not_null(dst_reg, src_reg);
} else {
__ decode_klass_not_null(dst_reg);
}
__ decode_klass_not_null(dst_reg, src_reg, rscratch1);
%}
ins_pipe(ialu_reg);
@ -14128,9 +14191,9 @@ instruct MoveL2D_reg_reg(vRegD dst, iRegL src) %{
// ============================================================================
// clearing of an array
instruct clearArray_reg_reg(iRegL_R11 cnt, iRegP_R10 base, Universe dummy, rFlagsReg cr)
instruct clearArray_reg_reg_immL0(iRegL_R11 cnt, iRegP_R10 base, immL0 zero, Universe dummy, rFlagsReg cr)
%{
match(Set dummy (ClearArray cnt base));
match(Set dummy (ClearArray (Binary cnt base) zero));
effect(USE_KILL cnt, USE_KILL base, KILL cr);
ins_cost(4 * INSN_COST);
@ -14147,11 +14210,28 @@ instruct clearArray_reg_reg(iRegL_R11 cnt, iRegP_R10 base, Universe dummy, rFlag
ins_pipe(pipe_class_memory);
%}
instruct clearArray_imm_reg(immL cnt, iRegP_R10 base, iRegL_R11 temp, Universe dummy, rFlagsReg cr)
instruct clearArray_reg_reg(iRegL_R11 cnt, iRegP_R10 base, iRegL val, Universe dummy, rFlagsReg cr)
%{
predicate((uint64_t)n->in(2)->get_long()
< (uint64_t)(BlockZeroingLowLimit >> LogBytesPerWord));
match(Set dummy (ClearArray cnt base));
predicate(((ClearArrayNode*)n)->word_copy_only());
match(Set dummy (ClearArray (Binary cnt base) val));
effect(USE_KILL cnt, USE_KILL base, KILL cr);
ins_cost(4 * INSN_COST);
format %{ "ClearArray $cnt, $base, $val" %}
ins_encode %{
__ fill_words($base$$Register, $cnt$$Register, $val$$Register);
%}
ins_pipe(pipe_class_memory);
%}
instruct clearArray_imm_reg(immL cnt, iRegP_R10 base, iRegL_R11 temp, immL0 zero, Universe dummy, rFlagsReg cr)
%{
predicate((uint64_t)n->in(2)->in(1)->get_long()
< (uint64_t)(BlockZeroingLowLimit >> LogBytesPerWord)
&& !((ClearArrayNode*)n)->word_copy_only());
match(Set dummy (ClearArray (Binary cnt base) zero));
effect(TEMP temp, USE_KILL base, KILL cr);
ins_cost(4 * INSN_COST);
@ -15467,8 +15547,28 @@ instruct CallLeafDirectVector(method meth)
// Call Runtime Instruction
// entry point is null, target holds the address to call
instruct CallLeafNoFPIndirect(iRegP target)
%{
predicate(n->as_Call()->entry_point() == nullptr);
match(CallLeafNoFP target);
ins_cost(CALL_COST);
format %{ "CALL, runtime leaf nofp indirect $target" %}
ins_encode %{
__ blr($target$$Register);
%}
ins_pipe(pipe_class_call);
%}
instruct CallLeafNoFPDirect(method meth)
%{
predicate(n->as_Call()->entry_point() != nullptr);
match(CallLeafNoFP);
effect(USE meth);

132
src/hotspot/cpu/aarch64/aarch64_atomic.ad
View File

@ -43,8 +43,7 @@ instruct compareAndExchangeB(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::byte, memory_order_release, $res$$Register);
__ sxtbw($res$$Register, $res$$Register);
%}
ins_pipe(pipe_slow);
@ -59,8 +58,7 @@ instruct compareAndExchangeS(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::halfword, memory_order_release, $res$$Register);
__ sxthw($res$$Register, $res$$Register);
%}
ins_pipe(pipe_slow);
@ -75,8 +73,7 @@ instruct compareAndExchangeI(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::word, memory_order_release, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -90,8 +87,7 @@ instruct compareAndExchangeL(iRegLNoSp res, indirect mem, iRegL oldval, iRegL ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::xword, memory_order_release, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -106,8 +102,7 @@ instruct compareAndExchangeN(iRegNNoSp res, indirect mem, iRegN oldval, iRegN ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::word, memory_order_release, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -122,8 +117,7 @@ instruct compareAndExchangeP(iRegPNoSp res, indirect mem, iRegP oldval, iRegP ne
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::xword, memory_order_release, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -138,8 +132,7 @@ instruct compareAndExchangeBAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::byte, memory_order_seq_cst, $res$$Register);
__ sxtbw($res$$Register, $res$$Register);
%}
ins_pipe(pipe_slow);
@ -155,8 +148,7 @@ instruct compareAndExchangeSAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::halfword, memory_order_seq_cst, $res$$Register);
__ sxthw($res$$Register, $res$$Register);
%}
ins_pipe(pipe_slow);
@ -172,8 +164,7 @@ instruct compareAndExchangeIAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::word, memory_order_seq_cst, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -188,8 +179,7 @@ instruct compareAndExchangeLAcq(iRegLNoSp res, indirect mem, iRegL oldval, iRegL
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::xword, memory_order_seq_cst, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -204,8 +194,7 @@ instruct compareAndExchangeNAcq(iRegNNoSp res, indirect mem, iRegN oldval, iRegN
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::word, memory_order_seq_cst, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -220,8 +209,7 @@ instruct compareAndExchangePAcq(iRegPNoSp res, indirect mem, iRegP oldval, iRegP
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::xword, memory_order_seq_cst, $res$$Register);
%}
ins_pipe(pipe_slow);
%}
@ -235,9 +223,7 @@ instruct compareAndSwapB(iRegINoSp res, indirect mem, iRegI oldval, iRegI newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::byte, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -252,9 +238,7 @@ instruct compareAndSwapS(iRegINoSp res, indirect mem, iRegI oldval, iRegI newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::halfword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -269,9 +253,7 @@ instruct compareAndSwapI(iRegINoSp res, indirect mem, iRegI oldval, iRegI newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -286,9 +268,7 @@ instruct compareAndSwapL(iRegINoSp res, indirect mem, iRegL oldval, iRegL newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -304,9 +284,7 @@ instruct compareAndSwapN(iRegINoSp res, indirect mem, iRegN oldval, iRegN newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -322,9 +300,7 @@ instruct compareAndSwapP(iRegINoSp res, indirect mem, iRegP oldval, iRegP newval
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -340,9 +316,7 @@ instruct compareAndSwapBAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::byte, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -358,9 +332,7 @@ instruct compareAndSwapSAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::halfword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -376,9 +348,7 @@ instruct compareAndSwapIAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -394,9 +364,7 @@ instruct compareAndSwapLAcq(iRegINoSp res, indirect mem, iRegL oldval, iRegL new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -412,9 +380,7 @@ instruct compareAndSwapNAcq(iRegINoSp res, indirect mem, iRegN oldval, iRegN new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -430,9 +396,7 @@ instruct compareAndSwapPAcq(iRegINoSp res, indirect mem, iRegP oldval, iRegP new
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ false, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -447,9 +411,7 @@ instruct weakCompareAndSwapB(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::byte, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -464,9 +426,7 @@ instruct weakCompareAndSwapS(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::halfword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -481,9 +441,7 @@ instruct weakCompareAndSwapI(iRegINoSp res, indirect mem, iRegI oldval, iRegI ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -498,9 +456,7 @@ instruct weakCompareAndSwapL(iRegINoSp res, indirect mem, iRegL oldval, iRegL ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -516,9 +472,7 @@ instruct weakCompareAndSwapN(iRegINoSp res, indirect mem, iRegN oldval, iRegN ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -534,9 +488,7 @@ instruct weakCompareAndSwapP(iRegINoSp res, indirect mem, iRegP oldval, iRegP ne
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ false, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_release);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -552,9 +504,7 @@ instruct weakCompareAndSwapBAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::byte, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::byte, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -570,9 +520,7 @@ instruct weakCompareAndSwapSAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::halfword, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::halfword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -588,9 +536,7 @@ instruct weakCompareAndSwapIAcq(iRegINoSp res, indirect mem, iRegI oldval, iRegI
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -606,9 +552,7 @@ instruct weakCompareAndSwapLAcq(iRegINoSp res, indirect mem, iRegL oldval, iRegL
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -624,9 +568,7 @@ instruct weakCompareAndSwapNAcq(iRegINoSp res, indirect mem, iRegN oldval, iRegN
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::word, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -642,9 +584,7 @@ instruct weakCompareAndSwapPAcq(iRegINoSp res, indirect mem, iRegP oldval, iRegP
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::xword, /*acquire*/ true, /*release*/ true,
/*weak*/ true, noreg);
__ cmpxchg_weak($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_seq_cst);
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);

14
src/hotspot/cpu/aarch64/aarch64_atomic_ad.m4
View File

@ -53,8 +53,7 @@ ifelse($7,Acq,INDENT(predicate(needs_acquiring_load_exclusive(n));),`dnl')
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::$4, /*acquire*/ ifelse($7,Acq,true,false), /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::$4, ifelse($7,Acq,memory_order_seq_cst,memory_order_release), $res$$Register);
__ $6($res$$Register, $res$$Register);
%}
ins_pipe(pipe_slow);
@ -76,8 +75,7 @@ ifelse($1$6,PAcq,INDENT(predicate(needs_acquiring_load_exclusive(n) && (n->as_Lo
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::$4, /*acquire*/ ifelse($6,Acq,true,false), /*release*/ true,
/*weak*/ false, $res$$Register);
Assembler::$4, ifelse($6,Acq,memory_order_seq_cst,memory_order_release), $res$$Register);
%}
ins_pipe(pipe_slow);
%}')dnl
@ -112,9 +110,7 @@ ifelse($6,Acq,INDENT(predicate(needs_acquiring_load_exclusive(n));),`dnl')
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::$4, /*acquire*/ ifelse($6,Acq,true,false), /*release*/ true,
/*weak*/ ifelse($7,Weak,true,false), noreg);
__ ifelse($7,Weak,cmpxchg_weak,cmpxchg)($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::$4, ifelse($6,Acq,memory_order_seq_cst,memory_order_release));
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);
@ -137,9 +133,7 @@ ifelse($1$6,PAcq,INDENT(predicate(needs_acquiring_load_exclusive(n) && (n->as_Lo
"csetw $res, EQ\t# $res <-- (EQ ? 1 : 0)"
%}
ins_encode %{
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register,
Assembler::$4, /*acquire*/ ifelse($6,Acq,true,false), /*release*/ true,
/*weak*/ ifelse($7,Weak,true,false), noreg);
__ ifelse($7,Weak,cmpxchg_weak,cmpxchg)($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::$4, ifelse($6,Acq,memory_order_seq_cst,memory_order_release));
__ csetw($res$$Register, Assembler::EQ);
%}
ins_pipe(pipe_slow);

185
src/hotspot/cpu/aarch64/aarch64_vector.ad
View File

@ -324,6 +324,17 @@ source %{
return false;
}
break;
case Op_DivVB:
case Op_DivVS:
case Op_DivVI:
case Op_DivVL:
// Integer vector divide is only available on SVE (SDIV for 32-bit and
// 64-bit elements). NEON has no integer vector divide instruction.
// BYTE/SHORT are widened to 32-bit, divided, and narrowed back.
if (UseSVE == 0) {
return false;
}
break;
default:
break;
}
@ -348,6 +359,11 @@ source %{
case Op_CompressBitsV:
case Op_ExpandBitsV:
case Op_VectorBitwiseBlend:
// There is no native SVE divide for BYTE/SHORT elements (these are
// emulated by widening to 32-bit), so the masked variants are handled
// by an unpredicated divide combined with a VectorBlend.
case Op_DivVB:
case Op_DivVS:
return false;
case Op_SaturatingAddV:
case Op_SaturatingSubV:
@ -1157,7 +1173,8 @@ instruct vmulI_sve(vReg dst_src1, vReg src2) %{
// vector mul - LONG
instruct vmulL_neon(vReg dst, vReg src1, vReg src2) %{
predicate(UseSVE == 0);
predicate(UseSVE == 0 && !n->as_MulVL()->has_int_inputs() &&
!n->as_MulVL()->has_uint_inputs());
match(Set dst (MulVL src1 src2));
format %{ "vmulL_neon $dst, $src1, $src2\t# 2L" %}
ins_encode %{
@ -1175,8 +1192,75 @@ instruct vmulL_neon(vReg dst, vReg src1, vReg src2) %{
ins_pipe(pipe_slow);
%}
// Specialization of vmulL_int_neon when both inputs are the same IR node
// (e.g. v * v). Avoids one redundant xtn and saves one temporary register.
instruct vmulL_int_neon_same(vReg dst, vReg src, vReg tmp) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_int_inputs() &&
n->in(1) == n->in(2));
match(Set dst (MulVL src src));
effect(TEMP tmp);
format %{ "vmulL_int_neon_same $dst, $src, $src\t# 2L. KILL $tmp" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp$$FloatRegister, __ T2S, $src$$FloatRegister, __ T2D);
__ smullv($dst$$FloatRegister, __ T2S, $tmp$$FloatRegister, $tmp$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vmulL_int_neon(vReg dst, vReg src1, vReg src2, vReg tmp1, vReg tmp2) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_int_inputs() &&
n->in(1) != n->in(2));
match(Set dst (MulVL src1 src2));
effect(TEMP tmp1, TEMP tmp2);
format %{ "vmulL_int_neon $dst, $src1, $src2\t# 2L. KILL $tmp1, $tmp2" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp1$$FloatRegister, __ T2S, $src1$$FloatRegister, __ T2D);
__ xtn($tmp2$$FloatRegister, __ T2S, $src2$$FloatRegister, __ T2D);
__ smullv($dst$$FloatRegister, __ T2S, $tmp1$$FloatRegister, $tmp2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
// Specialization of vmulL_uint_neon when both inputs are the same IR node
// (e.g. v * v). Avoids one redundant xtn and saves one temporary register.
instruct vmulL_uint_neon_same(vReg dst, vReg src, vReg tmp) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_uint_inputs() &&
n->in(1) == n->in(2));
match(Set dst (MulVL src src));
effect(TEMP tmp);
format %{ "vmulL_uint_neon_same $dst, $src, $src\t# 2L. KILL $tmp" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp$$FloatRegister, __ T2S, $src$$FloatRegister, __ T2D);
__ umullv($dst$$FloatRegister, __ T2S, $tmp$$FloatRegister, $tmp$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vmulL_uint_neon(vReg dst, vReg src1, vReg src2, vReg tmp1, vReg tmp2) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_uint_inputs() &&
n->in(1) != n->in(2));
match(Set dst (MulVL src1 src2));
effect(TEMP tmp1, TEMP tmp2);
format %{ "vmulL_uint_neon $dst, $src1, $src2\t# 2L. KILL $tmp1, $tmp2" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp1$$FloatRegister, __ T2S, $src1$$FloatRegister, __ T2D);
__ xtn($tmp2$$FloatRegister, __ T2S, $src2$$FloatRegister, __ T2D);
__ umullv($dst$$FloatRegister, __ T2S, $tmp1$$FloatRegister, $tmp2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vmulL_sve(vReg dst_src1, vReg src2) %{
predicate(UseSVE > 0);
predicate(UseSVE == 1 || (UseSVE == 2 && !n->as_MulVL()->has_int_inputs() &&
!n->as_MulVL()->has_uint_inputs()));
match(Set dst_src1 (MulVL dst_src1 src2));
format %{ "vmulL_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
@ -1185,6 +1269,26 @@ instruct vmulL_sve(vReg dst_src1, vReg src2) %{
ins_pipe(pipe_slow);
%}
instruct vmulL_int_sve2(vReg dst, vReg src1, vReg src2) %{
predicate(UseSVE == 2 && n->as_MulVL()->has_int_inputs());
match(Set dst (MulVL src1 src2));
format %{ "vmulL_int_sve2 $dst, $src1, $src2" %}
ins_encode %{
__ sve_smullb($dst$$FloatRegister, __ D, $src1$$FloatRegister, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vmulL_uint_sve2(vReg dst, vReg src1, vReg src2) %{
predicate(UseSVE == 2 && n->as_MulVL()->has_uint_inputs());
match(Set dst (MulVL src1 src2));
format %{ "vmulL_uint_sve2 $dst, $src1, $src2" %}
ins_encode %{
__ sve_umullb($dst$$FloatRegister, __ D, $src1$$FloatRegister, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
// vector mul - floating-point
instruct vmulHF(vReg dst, vReg src1, vReg src2) %{
@ -1389,6 +1493,83 @@ instruct vdivD_masked(vReg dst_src1, vReg src2, pRegGov pg) %{
ins_pipe(pipe_slow);
%}
// ------------------------------ Vector integer div ---------------------------
// BYTE and SHORT have no native integer divide on SVE (SDIV only supports 32-bit
// and 64-bit elements), so they are emulated by widening each element to 32-bit,
// performing SDIV, and narrowing the result back.
instruct vdivB_sve(vReg dst_src1, vReg src2, vReg vtmp1, vReg vtmp2, vReg vtmp3, vReg vtmp4) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVB dst_src1 src2));
effect(TEMP_DEF dst_src1, TEMP vtmp1, TEMP vtmp2, TEMP vtmp3, TEMP vtmp4);
format %{ "vdivB_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv_byte($dst_src1$$FloatRegister, $src2$$FloatRegister,
$vtmp1$$FloatRegister, $vtmp2$$FloatRegister,
$vtmp3$$FloatRegister, $vtmp4$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vdivS_sve(vReg dst_src1, vReg src2, vReg vtmp1, vReg vtmp2) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVS dst_src1 src2));
effect(TEMP_DEF dst_src1, TEMP vtmp1, TEMP vtmp2);
format %{ "vdivS_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv_short($dst_src1$$FloatRegister, $src2$$FloatRegister,
$vtmp1$$FloatRegister, $vtmp2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vdivI_sve(vReg dst_src1, vReg src2) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVI dst_src1 src2));
format %{ "vdivI_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv($dst_src1$$FloatRegister, __ S, ptrue, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vdivL_sve(vReg dst_src1, vReg src2) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVL dst_src1 src2));
format %{ "vdivL_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv($dst_src1$$FloatRegister, __ D, ptrue, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
// Vector integer div - predicated
//
// There is no native SVE divide for BYTE/SHORT elements (these are emulated by
// widening to 32-bit), so the masked variants are handled by an unpredicated
// divide combined with a VectorBlend.
instruct vdivI_masked(vReg dst_src1, vReg src2, pRegGov pg) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVI (Binary dst_src1 src2) pg));
format %{ "vdivI_masked $dst_src1, $pg, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv($dst_src1$$FloatRegister, __ S, $pg$$PRegister, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vdivL_masked(vReg dst_src1, vReg src2, pRegGov pg) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVL (Binary dst_src1 src2) pg));
format %{ "vdivL_masked $dst_src1, $pg, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv($dst_src1$$FloatRegister, __ D, $pg$$PRegister, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
// ------------------------------ Vector and -----------------------------------
// vector and

130
src/hotspot/cpu/aarch64/aarch64_vector_ad.m4
View File

@ -314,6 +314,17 @@ source %{
return false;
}
break;
case Op_DivVB:
case Op_DivVS:
case Op_DivVI:
case Op_DivVL:
// Integer vector divide is only available on SVE (SDIV for 32-bit and
// 64-bit elements). NEON has no integer vector divide instruction.
// BYTE/SHORT are widened to 32-bit, divided, and narrowed back.
if (UseSVE == 0) {
return false;
}
break;
default:
break;
}
@ -338,6 +349,11 @@ source %{
case Op_CompressBitsV:
case Op_ExpandBitsV:
case Op_VectorBitwiseBlend:
// There is no native SVE divide for BYTE/SHORT elements (these are
// emulated by widening to 32-bit), so the masked variants are handled
// by an unpredicated divide combined with a VectorBlend.
case Op_DivVB:
case Op_DivVS:
return false;
case Op_SaturatingAddV:
case Op_SaturatingSubV:
@ -736,7 +752,8 @@ BINARY_OP_NEON_SVE_PAIRWISE(vmulI, MulVI, mulv, sve_mul, S)
// vector mul - LONG
instruct vmulL_neon(vReg dst, vReg src1, vReg src2) %{
predicate(UseSVE == 0);
predicate(UseSVE == 0 && !n->as_MulVL()->has_int_inputs() &&
!n->as_MulVL()->has_uint_inputs());
match(Set dst (MulVL src1 src2));
format %{ "vmulL_neon $dst, $src1, $src2\t# 2L" %}
ins_encode %{
@ -754,8 +771,47 @@ instruct vmulL_neon(vReg dst, vReg src1, vReg src2) %{
ins_pipe(pipe_slow);
%}
dnl VMUL_L_NEON($1, $2 )
dnl VMUL_L_NEON(kind, insn )
define(`VMUL_L_NEON', `dnl
// Specialization of vmulL_$1_neon when both inputs are the same IR node
// (e.g. v * v). Avoids one redundant xtn and saves one temporary register.
instruct vmulL_$1_neon_same(vReg dst, vReg src, vReg tmp) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_$1_inputs() &&
n->in(1) == n->in(2));
match(Set dst (MulVL src src));
effect(TEMP tmp);
format %{ "vmulL_$1_neon_same $dst, $src, $src\t# 2L. KILL $tmp" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp$$FloatRegister, __ T2S, $src$$FloatRegister, __ T2D);
__ $2($dst$$FloatRegister, __ T2S, $tmp$$FloatRegister, $tmp$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vmulL_$1_neon(vReg dst, vReg src1, vReg src2, vReg tmp1, vReg tmp2) %{
predicate(UseSVE == 0 && n->as_MulVL()->has_$1_inputs() &&
n->in(1) != n->in(2));
match(Set dst (MulVL src1 src2));
effect(TEMP tmp1, TEMP tmp2);
format %{ "vmulL_$1_neon $dst, $src1, $src2\t# 2L. KILL $tmp1, $tmp2" %}
ins_encode %{
uint length_in_bytes = Matcher::vector_length_in_bytes(this);
assert(length_in_bytes == 16, "must be");
__ xtn($tmp1$$FloatRegister, __ T2S, $src1$$FloatRegister, __ T2D);
__ xtn($tmp2$$FloatRegister, __ T2S, $src2$$FloatRegister, __ T2D);
__ $2($dst$$FloatRegister, __ T2S, $tmp1$$FloatRegister, $tmp2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
')dnl
VMUL_L_NEON(int, smullv)
VMUL_L_NEON(uint, umullv)
instruct vmulL_sve(vReg dst_src1, vReg src2) %{
predicate(UseSVE > 0);
predicate(UseSVE == 1 || (UseSVE == 2 && !n->as_MulVL()->has_int_inputs() &&
!n->as_MulVL()->has_uint_inputs()));
match(Set dst_src1 (MulVL dst_src1 src2));
format %{ "vmulL_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
@ -764,6 +820,21 @@ instruct vmulL_sve(vReg dst_src1, vReg src2) %{
ins_pipe(pipe_slow);
%}
dnl VMUL_L_SVE2($1, $2 )
dnl VMUL_L_SVE2(kind, sve2_insn )
define(`VMUL_L_SVE2', `dnl
instruct vmulL_$1_sve2(vReg dst, vReg src1, vReg src2) %{
predicate(UseSVE == 2 && n->as_MulVL()->has_$1_inputs());
match(Set dst (MulVL src1 src2));
format %{ "vmulL_$1_sve2 $dst, $src1, $src2" %}
ins_encode %{
__ $2($dst$$FloatRegister, __ D, $src1$$FloatRegister, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
')dnl
VMUL_L_SVE2(int, sve_smullb)
VMUL_L_SVE2(uint, sve_umullb)
// vector mul - floating-point
BINARY_OP(vmulHF, MulVHF, fmul, sve_fmul, H)
BINARY_OP(vmulF, MulVF, fmul, sve_fmul, S)
@ -787,6 +858,61 @@ BINARY_OP_NEON_SVE_PAIRWISE(vdivD, DivVD, fdiv, sve_fdiv, D)
// vector float div - predicated
BINARY_OP_PREDICATE(vdivF, DivVF, sve_fdiv, S)
BINARY_OP_PREDICATE(vdivD, DivVD, sve_fdiv, D)
dnl
dnl BINARY_OP_SVE_ONLY($1, $2, $3, $4 )
dnl BINARY_OP_SVE_ONLY(rule_name, op_name, insn, size)
define(`BINARY_OP_SVE_ONLY', `
instruct $1_sve(vReg dst_src1, vReg src2) %{
predicate(UseSVE > 0);
match(Set dst_src1 ($2 dst_src1 src2));
format %{ "$1_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ $3($dst_src1$$FloatRegister, __ $4, ptrue, $src2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}')dnl
dnl
// ------------------------------ Vector integer div ---------------------------
// BYTE and SHORT have no native integer divide on SVE (SDIV only supports 32-bit
// and 64-bit elements), so they are emulated by widening each element to 32-bit,
// performing SDIV, and narrowing the result back.
instruct vdivB_sve(vReg dst_src1, vReg src2, vReg vtmp1, vReg vtmp2, vReg vtmp3, vReg vtmp4) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVB dst_src1 src2));
effect(TEMP_DEF dst_src1, TEMP vtmp1, TEMP vtmp2, TEMP vtmp3, TEMP vtmp4);
format %{ "vdivB_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv_byte($dst_src1$$FloatRegister, $src2$$FloatRegister,
$vtmp1$$FloatRegister, $vtmp2$$FloatRegister,
$vtmp3$$FloatRegister, $vtmp4$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
instruct vdivS_sve(vReg dst_src1, vReg src2, vReg vtmp1, vReg vtmp2) %{
predicate(UseSVE > 0);
match(Set dst_src1 (DivVS dst_src1 src2));
effect(TEMP_DEF dst_src1, TEMP vtmp1, TEMP vtmp2);
format %{ "vdivS_sve $dst_src1, $dst_src1, $src2" %}
ins_encode %{
__ sve_sdiv_short($dst_src1$$FloatRegister, $src2$$FloatRegister,
$vtmp1$$FloatRegister, $vtmp2$$FloatRegister);
%}
ins_pipe(pipe_slow);
%}
BINARY_OP_SVE_ONLY(vdivI, DivVI, sve_sdiv, S)
BINARY_OP_SVE_ONLY(vdivL, DivVL, sve_sdiv, D)
// Vector integer div - predicated
//
// There is no native SVE divide for BYTE/SHORT elements (these are emulated by
// widening to 32-bit), so the masked variants are handled by an unpredicated
// divide combined with a VectorBlend.
BINARY_OP_PREDICATE(vdivI, DivVI, sve_sdiv, S)
BINARY_OP_PREDICATE(vdivL, DivVL, sve_sdiv, D)
dnl
dnl BITWISE_OP_IMM($1, $2, $3, $4, $5, $6 )
dnl BITWISE_OP_IMM(rule_name, type, op_name, insn, size, basic_type)

54
src/hotspot/cpu/aarch64/assembler_aarch64.hpp
View File

@ -3475,13 +3475,15 @@ private:
public:
// SVE integer arithmetic - predicate
// SVE Arithmetic - Predicated
#define INSN(NAME, op1, op2) \
void NAME(FloatRegister Zdn_or_Zd_or_Vd, SIMD_RegVariant T, PRegister Pg, FloatRegister Znm_or_Vn) { \
assert(T != Q, "invalid register variant"); \
assert(ALLOWED, "invalid register variant"); \
sve_predicate_reg_insn(op1, op2, Zdn_or_Zd_or_Vd, T, Pg, Znm_or_Vn); \
}
// SVE Integer Arithmetic - Predicated (B/H/S/D element sizes).
#define ALLOWED (T != Q)
INSN(sve_abs, 0b00000100, 0b010110101); // vector abs, unary
INSN(sve_add, 0b00000100, 0b000000000); // vector add
INSN(sve_and, 0b00000100, 0b011010000); // vector and
@ -3511,15 +3513,16 @@ public:
INSN(sve_umaxv, 0b00000100, 0b001001001); // unsigned maximum reduction to scalar
INSN(sve_umin, 0b00000100, 0b001011000); // unsigned minimum vectors
INSN(sve_uminv, 0b00000100, 0b001011001); // unsigned minimum reduction to scalar
#undef INSN
#undef ALLOWED
// SVE floating-point arithmetic - predicate
#define INSN(NAME, op1, op2) \
void NAME(FloatRegister Zd_or_Zdn_or_Vd, SIMD_RegVariant T, PRegister Pg, FloatRegister Zn_or_Zm) { \
assert(T == H || T == S || T == D, "invalid register variant"); \
sve_predicate_reg_insn(op1, op2, Zd_or_Zdn_or_Vd, T, Pg, Zn_or_Zm); \
}
// SVE Integer Binary Arithmetic - Predicated (S/D element sizes).
#define ALLOWED (T == S || T == D)
INSN(sve_sdiv, 0b00000100, 0b010100000); // signed divide
INSN(sve_udiv, 0b00000100, 0b010101000); // unsigned divide
#undef ALLOWED
// SVE Floating-point Arithmetic - Predicated (H/S/D element sizes).
#define ALLOWED (T == H || T == S || T == D)
INSN(sve_fabd, 0b01100101, 0b001000100); // floating-point absolute difference
INSN(sve_fabs, 0b00000100, 0b011100101);
INSN(sve_fadd, 0b01100101, 0b000000100);
@ -3537,9 +3540,18 @@ public:
INSN(sve_frintp, 0b01100101, 0b000001101); // floating-point round to integral value, toward plus infinity
INSN(sve_fsqrt, 0b01100101, 0b001101101);
INSN(sve_fsub, 0b01100101, 0b000001100);
#undef ALLOWED
// SVE2 Signed/Unsigned Saturating Add/Sub - Predicated (B/H/S/D element sizes).
#define ALLOWED (T != Q)
INSN(sve_sqadd, 0b01000100, 0b011000100); // signed saturating add
INSN(sve_sqsub, 0b01000100, 0b011010100); // signed saturating sub
INSN(sve_uqadd, 0b01000100, 0b011001100); // unsigned saturating add
INSN(sve_uqsub, 0b01000100, 0b011011100); // unsigned saturating sub
#undef ALLOWED
#undef INSN
// SVE multiple-add/sub - predicated
// SVE multiple-add/sub - predicated
#define INSN(NAME, op0, op1, op2) \
void NAME(FloatRegister Zda, SIMD_RegVariant T, PRegister Pg, FloatRegister Zn, FloatRegister Zm) { \
starti; \
@ -4331,18 +4343,20 @@ public:
INSN(sve_bsl, 0b001, 0b1); // Bitwise select
#undef INSN
// SVE2 saturating operations - predicate
#define INSN(NAME, op1, op2) \
void NAME(FloatRegister Zdn, SIMD_RegVariant T, PRegister Pg, FloatRegister Znm) { \
assert(T != Q, "invalid register variant"); \
sve_predicate_reg_insn(op1, op2, Zdn, T, Pg, Znm); \
// SVE2 widening integer multiply - vector
#define INSN(NAME, is_unsigned, is_top) \
void NAME(FloatRegister Zd, SIMD_RegVariant T, FloatRegister Zn, FloatRegister Zm) { \
starti; \
assert(T != B && T != Q, "invalid size"); \
int op = 0b011100 | (is_unsigned ? 0b10 : 0) | (is_top ? 0b1 : 0); \
f(0b01000101, 31, 24), f(T, 23, 22), f(0, 21), rf(Zm, 16); \
f(op, 15, 10), rf(Zn, 5), rf(Zd, 0); \
}
INSN(sve_sqadd, 0b01000100, 0b011000100); // signed saturating add
INSN(sve_sqsub, 0b01000100, 0b011010100); // signed saturating sub
INSN(sve_uqadd, 0b01000100, 0b011001100); // unsigned saturating add
INSN(sve_uqsub, 0b01000100, 0b011011100); // unsigned saturating sub
INSN(sve_umullb, /* is_unsigned */ true, /* is_top */ false); // Unsigned widening multiply of bottom elements
INSN(sve_umullt, /* is_unsigned */ true, /* is_top */ true ); // Unsigned widening multiply of top elements
INSN(sve_smullb, /* is_unsigned */ false, /* is_top */ false); // Signed widening multiply of bottom elements
INSN(sve_smullt, /* is_unsigned */ false, /* is_top */ true ); // Signed widening multiply of top elements
#undef INSN
Assembler(CodeBuffer* code) : AbstractAssembler(code) {

90
src/hotspot/cpu/aarch64/c1_CodeStubs_aarch64.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 1999, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 1999, 2026, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2014, Red Hat Inc. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -119,6 +119,72 @@ void DivByZeroStub::emit_code(LIR_Assembler* ce) {
#endif
}
// Implementation of LoadFlattenedArrayStub
LoadFlattenedArrayStub::LoadFlattenedArrayStub(LIR_Opr array, LIR_Opr index, LIR_Opr result, CodeEmitInfo* info) {
_array = array;
_index = index;
_result = result;
_scratch_reg = FrameMap::r0_oop_opr;
_info = new CodeEmitInfo(info);
}
void LoadFlattenedArrayStub::emit_code(LIR_Assembler* ce) {
assert(__ rsp_offset() == 0, "frame size should be fixed");
__ bind(_entry);
ce->store_parameter(_array->as_register(), 1);
ce->store_parameter(_index->as_register(), 0);
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_load_flat_array_id)));
ce->add_call_info_here(_info);
ce->verify_oop_map(_info);
if (_result->as_register() != r0) {
__ mov(_result->as_register(), r0);
}
__ b(_continuation);
}
// Implementation of StoreFlattenedArrayStub
StoreFlattenedArrayStub::StoreFlattenedArrayStub(LIR_Opr array, LIR_Opr index, LIR_Opr value, CodeEmitInfo* info) {
_array = array;
_index = index;
_value = value;
_scratch_reg = FrameMap::r0_oop_opr;
_info = new CodeEmitInfo(info);
}
void StoreFlattenedArrayStub::emit_code(LIR_Assembler* ce) {
assert(__ rsp_offset() == 0, "frame size should be fixed");
__ bind(_entry);
ce->store_parameter(_array->as_register(), 2);
ce->store_parameter(_index->as_register(), 1);
ce->store_parameter(_value->as_register(), 0);
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_store_flat_array_id)));
ce->add_call_info_here(_info);
ce->verify_oop_map(_info);
__ b(_continuation);
}
// Implementation of SubstitutabilityCheckStub
SubstitutabilityCheckStub::SubstitutabilityCheckStub(LIR_Opr left, LIR_Opr right, CodeEmitInfo* info) {
_left = left;
_right = right;
_scratch_reg = FrameMap::r0_oop_opr;
_info = new CodeEmitInfo(info);
}
void SubstitutabilityCheckStub::emit_code(LIR_Assembler* ce) {
assert(__ rsp_offset() == 0, "frame size should be fixed");
__ bind(_entry);
ce->store_parameter(_left->as_register(), 1);
ce->store_parameter(_right->as_register(), 0);
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_substitutability_check_id)));
ce->add_call_info_here(_info);
ce->verify_oop_map(_info);
__ b(_continuation);
}
// Implementation of NewInstanceStub
@ -176,11 +242,13 @@ void NewTypeArrayStub::emit_code(LIR_Assembler* ce) {
// Implementation of NewObjectArrayStub
NewObjectArrayStub::NewObjectArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result, CodeEmitInfo* info) {
NewObjectArrayStub::NewObjectArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result,
CodeEmitInfo* info, bool is_null_free) {
_klass_reg = klass_reg;
_result = result;
_length = length;
_info = new CodeEmitInfo(info);
_is_null_free = is_null_free;
}
@ -189,7 +257,13 @@ void NewObjectArrayStub::emit_code(LIR_Assembler* ce) {
__ bind(_entry);
assert(_length->as_register() == r19, "length must in r19,");
assert(_klass_reg->as_register() == r3, "klass_reg must in r3");
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_new_object_array_id)));
if (_is_null_free) {
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_new_null_free_array_id)));
} else {
__ far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_new_object_array_id)));
}
ce->add_call_info_here(_info);
ce->verify_oop_map(_info);
assert(_result->as_register() == r0, "result must in r0");
@ -199,6 +273,16 @@ void NewObjectArrayStub::emit_code(LIR_Assembler* ce) {
void MonitorEnterStub::emit_code(LIR_Assembler* ce) {
assert(__ rsp_offset() == 0, "frame size should be fixed");
__ bind(_entry);
if (_throw_ie_stub != nullptr) {
// When we come here, _obj_reg has already been checked to be non-null.
__ ldr(rscratch1, Address(_obj_reg->as_register(), oopDesc::mark_offset_in_bytes()));
__ mov(rscratch2, markWord::inline_type_pattern);
__ andr(rscratch1, rscratch1, rscratch2);
__ cmp(rscratch1, rscratch2);
__ br(Assembler::EQ, *_throw_ie_stub->entry());
}
ce->store_parameter(_obj_reg->as_register(), 1);
ce->store_parameter(_lock_reg->as_register(), 0);
StubId enter_id;

323
src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
View File

@ -33,13 +33,16 @@
#include "c1/c1_Runtime1.hpp"
#include "c1/c1_ValueStack.hpp"
#include "ci/ciArrayKlass.hpp"
#include "ci/ciInlineKlass.hpp"
#include "ci/ciInstance.hpp"
#include "ci/ciObjArrayKlass.hpp"
#include "code/aotCodeCache.hpp"
#include "code/compiledIC.hpp"
#include "gc/shared/collectedHeap.hpp"
#include "gc/shared/gc_globals.hpp"
#include "nativeInst_aarch64.hpp"
#include "oops/objArrayKlass.hpp"
#include "oops/oop.inline.hpp"
#include "runtime/frame.inline.hpp"
#include "runtime/sharedRuntime.hpp"
#include "runtime/stubRoutines.hpp"
@ -413,7 +416,7 @@ int LIR_Assembler::emit_unwind_handler() {
// remove the activation and dispatch to the unwind handler
__ block_comment("remove_frame and dispatch to the unwind handler");
__ remove_frame(initial_frame_size_in_bytes());
__ remove_frame(initial_frame_size_in_bytes(), needs_stack_repair());
__ far_jump(RuntimeAddress(Runtime1::entry_for(StubId::c1_unwind_exception_id)));
// Emit the slow path assembly
@ -452,21 +455,53 @@ int LIR_Assembler::emit_deopt_handler() {
return entry_offset;
}
void LIR_Assembler::add_debug_info_for_branch(address adr, CodeEmitInfo* info) {
_masm->code_section()->relocate(adr, relocInfo::poll_type);
int pc_offset = code_offset();
flush_debug_info(pc_offset);
info->record_debug_info(compilation()->debug_info_recorder(), pc_offset);
if (info->exception_handlers() != nullptr) {
compilation()->add_exception_handlers_for_pco(pc_offset, info->exception_handlers());
}
}
void LIR_Assembler::return_op(LIR_Opr result, C1SafepointPollStub* code_stub) {
assert(result->is_illegal() || !result->is_single_cpu() || result->as_register() == r0, "word returns are in r0,");
if (InlineTypeReturnedAsFields) {
// Check if we are returning a non-null inline type and load its fields into registers
ciType* return_type = compilation()->method()->return_type();
if (return_type->is_inlinetype()) {
ciInlineKlass* vk = return_type->as_inline_klass();
if (vk->can_be_returned_as_fields()) {
address unpack_handler = vk->unpack_handler();
assert(unpack_handler != nullptr, "must be");
__ far_call(RuntimeAddress(unpack_handler));
}
} else if (return_type->is_instance_klass() && (!return_type->is_loaded() || StressCallingConvention)) {
Label skip;
Label not_null;
__ cbnz(r0, not_null);
// Returned value is null, zero all return registers because they may belong to oop fields
__ mov(j_rarg1, zr);
__ mov(j_rarg2, zr);
__ mov(j_rarg3, zr);
__ mov(j_rarg4, zr);
__ mov(j_rarg5, zr);
__ mov(j_rarg6, zr);
__ mov(j_rarg7, zr);
__ b(skip);
__ bind(not_null);
// Check if we are returning a non-null inline type and load its fields into registers
__ test_oop_is_not_inline_type(r0, rscratch2, skip, /* can_be_null= */ false);
// Load fields from a buffered value with an inline class specific handler
__ load_klass(rscratch1 /*dst*/, r0 /*src*/, rscratch2 /*tmp*/);
__ ldr(rscratch1, Address(rscratch1, InlineKlass::adr_members_offset()));
__ ldr(rscratch1, Address(rscratch1, InlineKlass::unpack_handler_offset()));
// Unpack handler can be null if inline type is not scalarizable in returns
__ cbz(rscratch1, skip);
__ blr(rscratch1);
__ bind(skip);
}
// At this point, r0 points to the value object (for interpreter or C1 caller).
// The fields of the object are copied into registers (for C2 caller).
}
// Pop the stack before the safepoint code
__ remove_frame(initial_frame_size_in_bytes());
__ remove_frame(initial_frame_size_in_bytes(), needs_stack_repair());
if (StackReservedPages > 0 && compilation()->has_reserved_stack_access()) {
__ reserved_stack_check();
@ -478,6 +513,10 @@ void LIR_Assembler::return_op(LIR_Opr result, C1SafepointPollStub* code_stub) {
__ ret(lr);
}
int LIR_Assembler::store_inline_type_fields_to_buf(ciInlineKlass* vk) {
return (__ store_inline_type_fields_to_buf(vk, false));
}
int LIR_Assembler::safepoint_poll(LIR_Opr tmp, CodeEmitInfo* info) {
guarantee(info != nullptr, "Shouldn't be null");
__ get_polling_page(rscratch1, relocInfo::poll_type);
@ -537,10 +576,10 @@ void LIR_Assembler::const2reg(LIR_Opr src, LIR_Opr dest, LIR_PatchCode patch_cod
}
case T_OBJECT: {
if (patch_code == lir_patch_none) {
jobject2reg(c->as_jobject(), dest->as_register());
} else {
if (patch_code != lir_patch_none) {
jobject2reg_with_patching(dest->as_register(), info);
} else {
jobject2reg(c->as_jobject(), dest->as_register());
}
break;
}
@ -1013,6 +1052,21 @@ void LIR_Assembler::load_unordered(LIR_Address *from_addr, LIR_Opr dest,
}
}
void LIR_Assembler::move(LIR_Opr src, LIR_Opr dst) {
assert(dst->is_cpu_register(), "must be");
assert(dst->type() == src->type(), "must be");
if (src->is_cpu_register()) {
reg2reg(src, dst);
} else if (src->is_stack()) {
stack2reg(src, dst, dst->type());
} else if (src->is_constant()) {
const2reg(src, dst, lir_patch_none, nullptr);
} else {
ShouldNotReachHere();
}
}
void LIR_Assembler::load_volatile(LIR_Address *from_addr, LIR_Opr dest,
BasicType type, CodeEmitInfo* info) {
__ lea(rscratch1, as_Address(from_addr));
@ -1240,7 +1294,7 @@ void LIR_Assembler::emit_alloc_array(LIR_OpAllocArray* op) {
Register len = op->len()->as_register();
__ uxtw(len, len);
if (UseSlowPath ||
if (UseSlowPath || op->always_slow_path() ||
(!UseFastNewObjectArray && is_reference_type(op->type())) ||
(!UseFastNewTypeArray && !is_reference_type(op->type()))) {
__ b(*op->stub()->entry());
@ -1317,27 +1371,29 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L
assert_different_registers(obj, k_RInfo, klass_RInfo);
if (should_profile) {
Register mdo = klass_RInfo;
__ mov_metadata(mdo, md->constant_encoding());
Label not_null;
__ cbnz(obj, not_null);
// Object is null; update MDO and exit
Address data_addr
= __ form_address(rscratch2, mdo,
md->byte_offset_of_slot(data, DataLayout::flags_offset()),
0);
__ ldrb(rscratch1, data_addr);
__ orr(rscratch1, rscratch1, BitData::null_seen_byte_constant());
__ strb(rscratch1, data_addr);
__ b(*obj_is_null);
__ bind(not_null);
if (op->need_null_check()) {
if (should_profile) {
Register mdo = klass_RInfo;
__ mov_metadata(mdo, md->constant_encoding());
Label not_null;
__ cbnz(obj, not_null);
// Object is null; update MDO and exit
Address data_addr
= __ form_address(rscratch2, mdo,
md->byte_offset_of_slot(data, DataLayout::flags_offset()),
0);
__ ldrb(rscratch1, data_addr);
__ orr(rscratch1, rscratch1, BitData::null_seen_byte_constant());
__ strb(rscratch1, data_addr);
__ b(*obj_is_null);
__ bind(not_null);
Register recv = k_RInfo;
__ load_klass(recv, obj);
type_profile_helper(mdo, md, data, recv);
} else {
__ cbz(obj, *obj_is_null);
Register recv = k_RInfo;
__ load_klass(recv, obj, rscratch1);
type_profile_helper(mdo, md, data, recv);
} else {
__ cbz(obj, *obj_is_null);
}
}
if (!k->is_loaded()) {
@ -1348,17 +1404,18 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L
__ verify_oop(obj);
if (op->fast_check()) {
assert(!k->is_loaded() || !k->is_obj_array_klass(), "Use refined array for a direct pointer comparison");
// get object class
// not a safepoint as obj null check happens earlier
__ load_klass(rscratch1, obj);
__ cmp( rscratch1, k_RInfo);
__ load_klass(rscratch2, obj, rscratch1);
__ cmp( rscratch2, k_RInfo);
__ br(Assembler::NE, *failure_target);
// successful cast, fall through to profile or jump
} else {
// get object class
// not a safepoint as obj null check happens earlier
__ load_klass(klass_RInfo, obj);
__ load_klass(klass_RInfo, obj, rscratch1);
if (k->is_loaded()) {
// See if we get an immediate positive hit
__ ldr(rscratch1, Address(klass_RInfo, int64_t(k->super_check_offset())));
@ -1370,7 +1427,18 @@ void LIR_Assembler::emit_typecheck_helper(LIR_OpTypeCheck *op, Label* success, L
// See if we get an immediate positive hit
__ br(Assembler::EQ, *success_target);
// check for self
__ cmp(klass_RInfo, k_RInfo);
if (k->is_loaded() && k->is_obj_array_klass()) {
// For a direct pointer comparison, we need the refined array klass pointer
ciKlass* k_refined = ciObjArrayKlass::make(k->as_obj_array_klass()->element_klass());
if (!k_refined->is_loaded()) {
bailout("encountered unloaded_ciobjarrayklass due to out of memory error");
return;
}
__ mov_metadata(rscratch1, k_refined->constant_encoding());
__ cmp(klass_RInfo, rscratch1);
} else {
__ cmp(klass_RInfo, k_RInfo);
}
__ br(Assembler::EQ, *success_target);
__ stp(klass_RInfo, k_RInfo, Address(__ pre(sp, -2 * wordSize)));
@ -1443,15 +1511,15 @@ void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) {
__ bind(not_null);
Register recv = k_RInfo;
__ load_klass(recv, value);
__ load_klass(recv, value, rscratch1);
type_profile_helper(mdo, md, data, recv);
} else {
__ cbz(value, done);
}
add_debug_info_for_null_check_here(op->info_for_exception());
__ load_klass(k_RInfo, array);
__ load_klass(klass_RInfo, value);
__ load_klass(k_RInfo, array, rscratch1);
__ load_klass(klass_RInfo, value, rscratch1);
// get instance klass (it's already uncompressed)
__ ldr(k_RInfo, Address(k_RInfo, ObjArrayKlass::element_klass_offset()));
@ -1491,13 +1559,97 @@ void LIR_Assembler::emit_opTypeCheck(LIR_OpTypeCheck* op) {
}
}
void LIR_Assembler::emit_opFlattenedArrayCheck(LIR_OpFlattenedArrayCheck* op) {
// We are loading/storing from/to an array that *may* be a flat array (the
// declared type is Object[], abstract[], interface[] or VT.ref[]).
// If this array is a flat array, take the slow path.
__ test_flat_array_oop(op->array()->as_register(), op->tmp()->as_register(), *op->stub()->entry());
}
void LIR_Assembler::emit_opNullFreeArrayCheck(LIR_OpNullFreeArrayCheck* op) {
// We are storing into an array that *may* be null-free (the declared type is
// Object[], abstract[], interface[] or VT.ref[]).
Label test_mark_word;
Register tmp = op->tmp()->as_register();
__ ldr(tmp, Address(op->array()->as_register(), oopDesc::mark_offset_in_bytes()));
__ tst(tmp, markWord::unlocked_value);
__ br(Assembler::NE, test_mark_word);
__ load_prototype_header(tmp, op->array()->as_register());
__ bind(test_mark_word);
__ tst(tmp, markWord::null_free_array_bit_in_place);
}
void LIR_Assembler::emit_opSubstitutabilityCheck(LIR_OpSubstitutabilityCheck* op) {
Label L_oops_equal;
Label L_oops_not_equal;
Label L_end;
Register left = op->left()->as_register();
Register right = op->right()->as_register();
__ cmp(left, right);
__ br(Assembler::EQ, L_oops_equal);
// (1) Null check -- if one of the operands is null, the other must not be null (because
// the two references are not equal), so they are not substitutable,
__ cbz(left, L_oops_not_equal);
__ cbz(right, L_oops_not_equal);
ciKlass* left_klass = op->left_klass();
ciKlass* right_klass = op->right_klass();
// (2) Inline type check -- if either of the operands is not an inline type,
// they are not substitutable. We do this only if we are not sure that the
// operands are inline type
if ((left_klass == nullptr || right_klass == nullptr) ||// The klass is still unloaded, or came from a Phi node.
!left_klass->is_inlinetype() || !right_klass->is_inlinetype()) {
Register tmp1 = op->tmp1()->as_register();
Register tmp2 = op->tmp2()->as_register();
__ mov(tmp1, markWord::inline_type_pattern);
__ ldr(tmp2, Address(left, oopDesc::mark_offset_in_bytes()));
__ andr(tmp1, tmp1, tmp2);
__ ldr(tmp2, Address(right, oopDesc::mark_offset_in_bytes()));
__ andr(tmp1, tmp1, tmp2);
__ cmp(tmp1, (u1)markWord::inline_type_pattern);
__ br(Assembler::NE, L_oops_not_equal);
}
// (3) Same klass check: if the operands are of different klasses, they are not substitutable.
if (left_klass != nullptr && left_klass->is_inlinetype() && left_klass == right_klass) {
// No need to load klass -- the operands are statically known to be the same inline klass.
__ b(*op->stub()->entry());
} else {
Register tmp1 = op->tmp1()->as_register();
Register tmp2 = op->tmp2()->as_register();
__ cmp_klasses_from_objects(left, right, tmp1, tmp2);
__ br(Assembler::EQ, *op->stub()->entry()); // same klass -> do slow check
// fall through to L_oops_not_equal
}
__ bind(L_oops_not_equal);
move(op->not_equal_result(), op->result_opr());
__ b(L_end);
// We've returned from the stub. R0 contains 0x0 IFF the two
// operands are not substitutable. (Don't compare against 0x1 in case the
// C compiler is naughty)
__ bind(*op->stub()->continuation());
__ cbz(r0, L_oops_not_equal); // (call_stub() == 0x0) -> not_equal
__ bind(L_oops_equal);
move(op->equal_result(), op->result_opr()); // (call_stub() != 0x0) -> equal
// fall-through
__ bind(L_end);
}
void LIR_Assembler::casw(Register addr, Register newval, Register cmpval) {
__ cmpxchg(addr, cmpval, newval, Assembler::word, /* acquire*/ true, /* release*/ true, /* weak*/ false, rscratch1);
__ cmpxchg(addr, cmpval, newval, Assembler::word, memory_order_seq_cst, rscratch1);
__ cset(rscratch1, Assembler::NE);
}
void LIR_Assembler::casl(Register addr, Register newval, Register cmpval) {
__ cmpxchg(addr, cmpval, newval, Assembler::xword, /* acquire*/ true, /* release*/ true, /* weak*/ false, rscratch1);
__ cmpxchg(addr, cmpval, newval, Assembler::xword, memory_order_seq_cst, rscratch1);
__ cset(rscratch1, Assembler::NE);
}
@ -2002,7 +2154,7 @@ void LIR_Assembler::call(LIR_OpJavaCall* op, relocInfo::relocType rtype) {
bailout("trampoline stub overflow");
return;
}
add_call_info(code_offset(), op->info());
add_call_info(code_offset(), op->info(), op->maybe_return_as_fields());
__ post_call_nop();
}
@ -2013,7 +2165,7 @@ void LIR_Assembler::ic_call(LIR_OpJavaCall* op) {
bailout("trampoline stub overflow");
return;
}
add_call_info(code_offset(), op->info());
add_call_info(code_offset(), op->info(), op->maybe_return_as_fields());
__ post_call_nop();
}
@ -2176,6 +2328,17 @@ void LIR_Assembler::store_parameter(jobject o, int offset_from_rsp_in_words) {
__ str(rscratch1, Address(sp, offset_from_rsp_in_bytes));
}
void LIR_Assembler::arraycopy_inlinetype_check(Register obj, Register tmp, CodeStub* slow_path, bool is_dest, bool null_check) {
if (null_check) {
__ cbz(obj, *slow_path->entry());
}
if (is_dest) {
__ test_null_free_array_oop(obj, tmp, *slow_path->entry());
__ test_flat_array_oop(obj, tmp, *slow_path->entry());
} else {
__ test_flat_array_oop(obj, tmp, *slow_path->entry());
}
}
// This code replaces a call to arraycopy; no exception may
// be thrown in this code, they must be thrown in the System.arraycopy
@ -2194,6 +2357,12 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
BasicType basic_type = default_type != nullptr ? default_type->element_type()->basic_type() : T_ILLEGAL;
if (is_reference_type(basic_type)) basic_type = T_OBJECT;
if (flags & LIR_OpArrayCopy::always_slow_path) {
__ b(*stub->entry());
__ bind(*stub->continuation());
return;
}
// if we don't know anything, just go through the generic arraycopy
if (default_type == nullptr // || basic_type == T_OBJECT
) {
@ -2247,6 +2416,14 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
return;
}
// Handle inline type arrays
if (flags & LIR_OpArrayCopy::src_inlinetype_check) {
arraycopy_inlinetype_check(src, tmp, stub, false, (flags & LIR_OpArrayCopy::src_null_check));
}
if (flags & LIR_OpArrayCopy::dst_inlinetype_check) {
arraycopy_inlinetype_check(dst, tmp, stub, true, (flags & LIR_OpArrayCopy::dst_null_check));
}
assert(default_type != nullptr && default_type->is_array_klass() && default_type->is_loaded(), "must be true at this point");
int elem_size = type2aelembytes(basic_type);
@ -2268,14 +2445,14 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
// an instance type.
if (flags & LIR_OpArrayCopy::type_check) {
if (!(flags & LIR_OpArrayCopy::LIR_OpArrayCopy::dst_objarray)) {
__ load_klass(tmp, dst);
__ load_klass(tmp, dst, rscratch1);
__ ldrw(rscratch1, Address(tmp, in_bytes(Klass::layout_helper_offset())));
__ cmpw(rscratch1, Klass::_lh_neutral_value);
__ br(Assembler::GE, *stub->entry());
}
if (!(flags & LIR_OpArrayCopy::LIR_OpArrayCopy::src_objarray)) {
__ load_klass(tmp, src);
__ load_klass(tmp, src, rscratch1);
__ ldrw(rscratch1, Address(tmp, in_bytes(Klass::layout_helper_offset())));
__ cmpw(rscratch1, Klass::_lh_neutral_value);
__ br(Assembler::GE, *stub->entry());
@ -2329,8 +2506,8 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
__ PUSH(src, dst);
__ load_klass(src, src);
__ load_klass(dst, dst);
__ load_klass(src, src, rscratch1);
__ load_klass(dst, dst, rscratch1);
__ check_klass_subtype_fast_path(src, dst, tmp, &cont, &slow, nullptr);
@ -2354,9 +2531,9 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
assert(flags & mask, "one of the two should be known to be an object array");
if (!(flags & LIR_OpArrayCopy::src_objarray)) {
__ load_klass(tmp, src);
__ load_klass(tmp, src, rscratch1);
} else if (!(flags & LIR_OpArrayCopy::dst_objarray)) {
__ load_klass(tmp, dst);
__ load_klass(tmp, dst, rscratch1);
}
int lh_offset = in_bytes(Klass::layout_helper_offset());
Address klass_lh_addr(tmp, lh_offset);
@ -2382,7 +2559,7 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
__ uxtw(c_rarg2, length);
assert_different_registers(c_rarg2, dst);
__ load_klass(c_rarg4, dst);
__ load_klass(c_rarg4, dst, rscratch1);
__ ldr(c_rarg4, Address(c_rarg4, ObjArrayKlass::element_klass_offset()));
__ ldrw(c_rarg3, Address(c_rarg4, Klass::super_check_offset_offset()));
__ far_call(RuntimeAddress(copyfunc_addr));
@ -2438,12 +2615,12 @@ void LIR_Assembler::emit_arraycopy(LIR_OpArrayCopy* op) {
__ mov_metadata(tmp, default_type->constant_encoding());
if (basic_type != T_OBJECT) {
__ cmp_klass(dst, tmp, rscratch1);
__ cmp_klass(dst, tmp, rscratch1, rscratch2);
__ br(Assembler::NE, halt);
__ cmp_klass(src, tmp, rscratch1);
__ cmp_klass(src, tmp, rscratch1, rscratch2);
__ br(Assembler::EQ, known_ok);
} else {
__ cmp_klass(dst, tmp, rscratch1);
__ cmp_klass(dst, tmp, rscratch1, rscratch2);
__ br(Assembler::EQ, known_ok);
__ cmp(src, dst);
__ br(Assembler::EQ, known_ok);
@ -2518,7 +2695,7 @@ void LIR_Assembler::emit_load_klass(LIR_OpLoadKlass* op) {
add_debug_info_for_null_check_here(info);
}
__ load_klass(result, obj);
__ load_klass(result, obj, rscratch1);
}
void LIR_Assembler::emit_profile_call(LIR_OpProfileCall* op) {
@ -2560,7 +2737,7 @@ void LIR_Assembler::emit_profile_call(LIR_OpProfileCall* op) {
// Fall back to runtime helper to handle the rest at runtime.
__ mov_metadata(recv, known_klass->constant_encoding());
} else {
__ load_klass(recv, recv);
__ load_klass(recv, recv, rscratch1);
}
type_profile_helper(mdo, md, data, recv);
} else {
@ -2646,7 +2823,7 @@ void LIR_Assembler::emit_profile_type(LIR_OpProfileType* op) {
#ifdef ASSERT
if (exact_klass != nullptr) {
Label ok;
__ load_klass(tmp, tmp);
__ load_klass(tmp, tmp, rscratch1);
__ mov_metadata(rscratch1, exact_klass->constant_encoding());
__ eor(rscratch1, tmp, rscratch1);
__ cbz(rscratch1, ok);
@ -2659,7 +2836,7 @@ void LIR_Assembler::emit_profile_type(LIR_OpProfileType* op) {
if (exact_klass != nullptr) {
__ mov_metadata(tmp, exact_klass->constant_encoding());
} else {
__ load_klass(tmp, tmp);
__ load_klass(tmp, tmp, rscratch1);
}
__ ldr(rscratch2, mdo_addr);
@ -2761,6 +2938,26 @@ void LIR_Assembler::emit_profile_type(LIR_OpProfileType* op) {
COMMENT("} emit_profile_type");
}
void LIR_Assembler::emit_profile_inline_type(LIR_OpProfileInlineType* op) {
Register obj = op->obj()->as_register();
Register tmp = op->tmp()->as_pointer_register();
bool not_null = op->not_null();
int flag = op->flag();
Label not_inline_type;
if (!not_null) {
__ cbz(obj, not_inline_type);
}
__ test_oop_is_not_inline_type(obj, tmp, not_inline_type);
Address mdo_addr = as_Address(op->mdp()->as_address_ptr(), rscratch2);
__ ldrb(rscratch1, mdo_addr);
__ orr(rscratch1, rscratch1, flag);
__ strb(rscratch1, mdo_addr);
__ bind(not_inline_type);
}
void LIR_Assembler::align_backward_branch_target() {
}
@ -2902,6 +3099,10 @@ void LIR_Assembler::get_thread(LIR_Opr result_reg) {
__ mov(result_reg->as_register(), rthread);
}
void LIR_Assembler::check_orig_pc() {
__ ldr(rscratch2, frame_map()->address_for_orig_pc_addr());
__ cmp(rscratch2, (u1)NULL_WORD);
}
void LIR_Assembler::peephole(LIR_List *lir) {
#if 0

6
src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2000, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2000, 2026, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2014, Red Hat Inc. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -52,7 +52,6 @@ friend class ArrayCopyStub;
// Record the type of the receiver in ReceiverTypeData
void type_profile_helper(Register mdo, ciMethodData *md,
ciProfileData *data, Register recv);
void add_debug_info_for_branch(address adr, CodeEmitInfo* info);
void casw(Register addr, Register newval, Register cmpval);
void casl(Register addr, Register newval, Register cmpval);
@ -79,6 +78,9 @@ friend class ArrayCopyStub;
_deopt_handler_size = 4 * NativeInstruction::instruction_size
};
void arraycopy_inlinetype_check(Register obj, Register tmp, CodeStub* slow_path, bool is_dest, bool null_check);
void move(LIR_Opr src, LIR_Opr dst);
public:
void store_parameter(Register r, int offset_from_esp_in_words);

44
src/hotspot/cpu/aarch64/c1_LIRGenerator_aarch64.cpp
View File

@ -32,6 +32,7 @@
#include "c1/c1_Runtime1.hpp"
#include "c1/c1_ValueStack.hpp"
#include "ci/ciArray.hpp"
#include "ci/ciInlineKlass.hpp"
#include "ci/ciObjArrayKlass.hpp"
#include "ci/ciTypeArrayKlass.hpp"
#include "compiler/compilerDefinitions.inline.hpp"
@ -323,11 +324,17 @@ void LIRGenerator::do_MonitorEnter(MonitorEnter* x) {
if (x->needs_null_check()) {
info_for_exception = state_for(x);
}
CodeStub* throw_ie_stub =
x->maybe_inlinetype() ?
new SimpleExceptionStub(StubId::c1_throw_identity_exception_id, obj.result(), state_for(x)) :
nullptr;
// this CodeEmitInfo must not have the xhandlers because here the
// object is already locked (xhandlers expect object to be unlocked)
CodeEmitInfo* info = state_for(x, x->state(), true);
monitor_enter(obj.result(), lock, syncTempOpr(), scratch,
x->monitor_no(), info_for_exception, info);
x->monitor_no(), info_for_exception, info, throw_ie_stub);
}
@ -1127,14 +1134,15 @@ void LIRGenerator::do_NewInstance(NewInstance* x) {
tty->print_cr(" ###class not loaded at new bci %d", x->printable_bci());
}
#endif
CodeEmitInfo* info = state_for(x, x->state());
CodeEmitInfo* info = state_for(x, x->needs_state_before() ? x->state_before() : x->state());
LIR_Opr reg = result_register_for(x->type());
new_instance(reg, x->klass(), x->is_unresolved(),
FrameMap::r10_oop_opr,
FrameMap::r11_oop_opr,
FrameMap::r4_oop_opr,
LIR_OprFact::illegalOpr,
FrameMap::r3_metadata_opr, info);
!x->is_unresolved() && x->klass()->is_inlinetype(),
FrameMap::r10_oop_opr,
FrameMap::r11_oop_opr,
FrameMap::r4_oop_opr,
LIR_OprFact::illegalOpr,
FrameMap::r3_metadata_opr, info);
LIR_Opr result = rlock_result(x);
__ move(reg, result);
}
@ -1190,13 +1198,20 @@ void LIRGenerator::do_NewObjectArray(NewObjectArray* x) {
length.load_item_force(FrameMap::r19_opr);
LIR_Opr len = length.result();
CodeStub* slow_path = new NewObjectArrayStub(klass_reg, len, reg, info);
ciKlass* obj = (ciKlass*) ciObjArrayKlass::make(x->klass());
ciKlass* obj = ciObjArrayKlass::make(x->klass());
// TODO 8265122 Implement a fast path for this
bool is_flat = obj->is_loaded() && obj->is_flat_array_klass();
bool is_null_free = obj->is_loaded() && obj->as_array_klass()->is_elem_null_free();
CodeStub* slow_path = new NewObjectArrayStub(klass_reg, len, reg, info, is_null_free);
if (obj == ciEnv::unloaded_ciobjarrayklass()) {
BAILOUT("encountered unloaded_ciobjarrayklass due to out of memory error");
}
klass2reg_with_patching(klass_reg, obj, patching_info);
__ allocate_array(reg, len, tmp1, tmp2, tmp3, tmp4, T_OBJECT, klass_reg, slow_path);
bool always_slow_path = is_null_free || is_flat;
__ allocate_array(reg, len, tmp1, tmp2, tmp3, tmp4, T_OBJECT, klass_reg, slow_path, true /*zero_array*/, always_slow_path);
LIR_Opr result = rlock_result(x);
__ move(reg, result);
@ -1291,7 +1306,7 @@ void LIRGenerator::do_CheckCast(CheckCast* x) {
__ checkcast(reg, obj.result(), x->klass(),
new_register(objectType), new_register(objectType), tmp3,
x->direct_compare(), info_for_exception, patching_info, stub,
x->profiled_method(), x->profiled_bci());
x->profiled_method(), x->profiled_bci(), x->is_null_free());
}
void LIRGenerator::do_InstanceOf(InstanceOf* x) {
@ -1372,7 +1387,12 @@ void LIRGenerator::do_If(If* x) {
__ safepoint(LIR_OprFact::illegalOpr, state_for(x, x->state_before()));
}
__ cmp(lir_cond(cond), left, right);
if (x->substitutability_check()) {
substitutability_check(x, *xin, *yin);
} else {
__ cmp(lir_cond(cond), left, right);
}
// Generate branch profiling. Profiling code doesn't kill flags.
profile_branch(x, cond);
move_to_phi(x->state());

1
src/hotspot/cpu/aarch64/c1_LIR_aarch64.cpp
View File

@ -49,5 +49,6 @@ void LIR_Address::verify() const {
assert(index()->is_illegal() || index()->is_double_cpu() || index()->is_single_cpu(), "wrong index operand");
assert(base()->type() == T_ADDRESS || base()->type() == T_OBJECT || base()->type() == T_LONG || base()->type() == T_METADATA,
"wrong type for addresses");
assert(index()->is_illegal() || disp() == 0, "cannot set both index and displacement");
}
#endif // PRODUCT

107
src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp
View File

@ -27,10 +27,13 @@
#include "c1/c1_Runtime1.hpp"
#include "gc/shared/barrierSetAssembler.hpp"
#include "gc/shared/collectedHeap.hpp"
#include "gc/shared/barrierSet.hpp"
#include "gc/shared/barrierSetAssembler.hpp"
#include "gc/shared/tlab_globals.hpp"
#include "interpreter/interpreter.hpp"
#include "oops/arrayOop.hpp"
#include "oops/markWord.hpp"
#include "runtime/arguments.hpp"
#include "runtime/basicLock.hpp"
#include "runtime/os.hpp"
#include "runtime/sharedRuntime.hpp"
@ -99,13 +102,22 @@ void C1_MacroAssembler::try_allocate(Register obj, Register var_size_in_bytes, i
void C1_MacroAssembler::initialize_header(Register obj, Register klass, Register len, Register t1, Register t2) {
assert_different_registers(obj, klass, len);
if (UseCompactObjectHeaders) {
if (UseCompactObjectHeaders || Arguments::is_valhalla_enabled()) {
// COH: Markword contains class pointer which is only known at runtime.
// Valhalla: Could have value class which has a different prototype header to a normal object.
// In both cases, we need to fetch dynamically.
ldr(t1, Address(klass, Klass::prototype_header_offset()));
str(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
} else {
// Otherwise: Can use the statically computed prototype header which is the same for every object.
mov(t1, checked_cast<int32_t>(markWord::prototype().value()));
str(t1, Address(obj, oopDesc::mark_offset_in_bytes()));
encode_klass_not_null(t1, klass); // Take care not to kill klass
}
if (!UseCompactObjectHeaders) {
// COH: Markword already contains class pointer. Nothing else to do.
// Otherwise: Store encoded klass pointer following the markword
encode_klass_not_null(t1, klass, t1); // Take care not to kill klass
strw(t1, Address(obj, oopDesc::klass_offset_in_bytes()));
}
@ -237,20 +249,37 @@ void C1_MacroAssembler::allocate_array(Register obj, Register len, Register t1,
verify_oop(obj);
}
void C1_MacroAssembler::build_frame(int framesize, int bang_size_in_bytes) {
assert(bang_size_in_bytes >= framesize, "stack bang size incorrect");
void C1_MacroAssembler::build_frame_helper(int frame_size_in_bytes, int sp_offset_for_orig_pc, int sp_inc, bool reset_orig_pc, bool needs_stack_repair) {
MacroAssembler::build_frame(frame_size_in_bytes);
if (needs_stack_repair) {
save_stack_increment(sp_inc, frame_size_in_bytes);
}
if (reset_orig_pc) {
// Zero orig_pc to detect deoptimization during buffering in the entry points
str(zr, Address(sp, sp_offset_for_orig_pc));
}
}
void C1_MacroAssembler::build_frame(int frame_size_in_bytes, int bang_size_in_bytes,
int sp_offset_for_orig_pc,
bool needs_stack_repair, bool has_scalarized_args,
Label* verified_inline_entry_label) {
// Make sure there is enough stack space for this method's activation.
// Note that we do this before creating a frame.
assert(bang_size_in_bytes >= frame_size_in_bytes, "stack bang size incorrect");
generate_stack_overflow_check(bang_size_in_bytes);
MacroAssembler::build_frame(framesize);
build_frame_helper(frame_size_in_bytes, sp_offset_for_orig_pc, 0, has_scalarized_args, needs_stack_repair);
// Insert nmethod entry barrier into frame.
BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler();
bs->nmethod_entry_barrier(this, nullptr /* slow_path */, nullptr /* continuation */, nullptr /* guard */);
}
void C1_MacroAssembler::remove_frame(int framesize) {
MacroAssembler::remove_frame(framesize);
if (verified_inline_entry_label != nullptr) {
// Jump here from the scalarized entry points that already created the frame.
bind(*verified_inline_entry_label);
}
}
@ -260,8 +289,70 @@ void C1_MacroAssembler::verified_entry(bool breakAtEntry) {
// must ensure that this first instruction is a B, BL, NOP, BKPT,
// SVC, HVC, or SMC. Make it a NOP.
nop();
if (C1Breakpoint) brk(1);
}
int C1_MacroAssembler::scalarized_entry(const CompiledEntrySignature* ces, int frame_size_in_bytes, int bang_size_in_bytes, int sp_offset_for_orig_pc, Label& verified_inline_entry_label, bool is_inline_ro_entry) {
assert(InlineTypePassFieldsAsArgs, "sanity");
// Make sure there is enough stack space for this method's activation.
assert(bang_size_in_bytes >= frame_size_in_bytes, "stack bang size incorrect");
generate_stack_overflow_check(bang_size_in_bytes);
GrowableArray<SigEntry>* sig = ces->sig();
GrowableArray<SigEntry>* sig_cc = is_inline_ro_entry ? ces->sig_cc_ro() : ces->sig_cc();
VMRegPair* regs = ces->regs();
VMRegPair* regs_cc = is_inline_ro_entry ? ces->regs_cc_ro() : ces->regs_cc();
int args_on_stack = ces->args_on_stack();
int args_on_stack_cc = is_inline_ro_entry ? ces->args_on_stack_cc_ro() : ces->args_on_stack_cc();
assert(sig->length() <= sig_cc->length(), "Zero-sized inline class not allowed!");
BasicType* sig_bt = NEW_RESOURCE_ARRAY(BasicType, sig_cc->length());
int args_passed = sig->length();
int args_passed_cc = SigEntry::fill_sig_bt(sig_cc, sig_bt);
// Create a temp frame so we can call into the runtime. It must be properly set up to accommodate GC.
build_frame_helper(frame_size_in_bytes, sp_offset_for_orig_pc, 0, true, ces->c1_needs_stack_repair());
// The runtime call might safepoint, make sure nmethod entry barrier is executed
BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler();
// C1 code is not hot enough to micro optimize the nmethod entry barrier with an out-of-line stub
bs->nmethod_entry_barrier(this, nullptr /* slow_path */, nullptr /* continuation */, nullptr /* guard */);
mov(r19, (intptr_t) ces->method());
if (is_inline_ro_entry) {
far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_buffer_inline_args_no_receiver_id)));
} else {
far_call(RuntimeAddress(Runtime1::entry_for(StubId::c1_buffer_inline_args_id)));
}
int rt_call_offset = offset();
// The runtime call returns the new array in r20 instead of the usual r0
// because r0 is also j_rarg7 which may be holding a live argument here.
Register val_array = r20;
// Remove the temp frame
MacroAssembler::remove_frame(frame_size_in_bytes);
// Check if we need to extend the stack for packing
int sp_inc = 0;
if (args_on_stack > args_on_stack_cc) {
sp_inc = extend_stack_for_inline_args(args_on_stack);
}
shuffle_inline_args(true, is_inline_ro_entry, sig_cc,
args_passed_cc, args_on_stack_cc, regs_cc, // from
args_passed, args_on_stack, regs, // to
sp_inc, val_array);
// Create the real frame. Below jump will then skip over the stack banging and frame
// setup code in the verified_inline_entry (which has a different real_frame_size).
build_frame_helper(frame_size_in_bytes, sp_offset_for_orig_pc, sp_inc, false, ces->c1_needs_stack_repair());
b(verified_inline_entry_label);
return rt_call_offset;
}
void C1_MacroAssembler::load_parameter(int offset_in_words, Register reg) {
// rfp, + 0: link
// + 1: return address

144
src/hotspot/cpu/aarch64/c1_Runtime1_aarch64.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 1999, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 1999, 2026, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2014, 2021, Red Hat Inc. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -741,6 +741,7 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
case StubId::c1_new_type_array_id:
case StubId::c1_new_object_array_id:
case StubId::c1_new_null_free_array_id:
{
Register length = r19; // Incoming
Register klass = r3; // Incoming
@ -748,8 +749,10 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
if (id == StubId::c1_new_type_array_id) {
__ set_info("new_type_array", dont_gc_arguments);
} else {
} else if (id == StubId::c1_new_object_array_id) {
__ set_info("new_object_array", dont_gc_arguments);
} else {
__ set_info("new_null_free_array", dont_gc_arguments);
}
#ifdef ASSERT
@ -759,13 +762,28 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
Register t0 = obj;
__ ldrw(t0, Address(klass, Klass::layout_helper_offset()));
__ asrw(t0, t0, Klass::_lh_array_tag_shift);
int tag = ((id == StubId::c1_new_type_array_id)
? Klass::_lh_array_tag_type_value
: Klass::_lh_array_tag_obj_value);
__ mov(rscratch1, tag);
__ cmpw(t0, rscratch1);
__ br(Assembler::EQ, ok);
__ stop("assert(is an array klass)");
switch (id) {
case StubId::c1_new_type_array_id:
__ cmpw(t0, Klass::_lh_array_tag_type_value);
__ br(Assembler::EQ, ok);
__ stop("assert(is a type array klass)");
break;
case StubId::c1_new_object_array_id:
__ cmpw(t0, Klass::_lh_array_tag_ref_value); // new "[Ljava/lang/Object;"
__ br(Assembler::EQ, ok);
__ cmpw(t0, Klass::_lh_array_tag_flat_value); // new "[LVT;"
__ br(Assembler::EQ, ok);
__ stop("assert(is an object or inline type array klass)");
break;
case StubId::c1_new_null_free_array_id:
__ cmpw(t0, Klass::_lh_array_tag_flat_value); // the array can be a flat array.
__ br(Assembler::EQ, ok);
__ cmpw(t0, Klass::_lh_array_tag_ref_value); // the array cannot be a flat array (due to the InlineArrayElementMaxFlatSize, etc.)
__ br(Assembler::EQ, ok);
__ stop("assert(is an object or inline type array klass)");
break;
default: ShouldNotReachHere();
}
__ should_not_reach_here();
__ bind(ok);
}
@ -776,8 +794,11 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
int call_offset;
if (id == StubId::c1_new_type_array_id) {
call_offset = __ call_RT(obj, noreg, CAST_FROM_FN_PTR(address, new_type_array), klass, length);
} else {
} else if (id == StubId::c1_new_object_array_id) {
call_offset = __ call_RT(obj, noreg, CAST_FROM_FN_PTR(address, new_object_array), klass, length);
} else {
assert(id == StubId::c1_new_null_free_array_id, "must be");
call_offset = __ call_RT(obj, noreg, CAST_FROM_FN_PTR(address, new_null_free_array), klass, length);
}
oop_maps = new OopMapSet();
@ -812,6 +833,89 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
}
break;
case StubId::c1_buffer_inline_args_id:
case StubId::c1_buffer_inline_args_no_receiver_id:
{
const char* name = (id == StubId::c1_buffer_inline_args_id) ?
"buffer_inline_args" : "buffer_inline_args_no_receiver";
StubFrame f(sasm, name, dont_gc_arguments);
OopMap* map = save_live_registers(sasm);
Register method = r19; // Incoming
address entry = (id == StubId::c1_buffer_inline_args_id) ?
CAST_FROM_FN_PTR(address, buffer_inline_args) :
CAST_FROM_FN_PTR(address, buffer_inline_args_no_receiver);
// This is called from a C1 method's scalarized entry point
// where r0-r7 may be holding live argument values so we can't
// return the result in r0 as the other stubs do. LR is used as
// a temporary below to avoid the result being clobbered by
// restore_live_registers. It's saved and restored by
// StubAssembler::prologue and epilogue anyway.
int call_offset = __ call_RT(lr, noreg, entry, method);
oop_maps = new OopMapSet();
oop_maps->add_gc_map(call_offset, map);
restore_live_registers(sasm);
__ mov(r20, lr);
__ verify_oop(r20); // r20: an array of buffered value objects
}
break;
case StubId::c1_load_flat_array_id:
{
StubFrame f(sasm, "load_flat_array", dont_gc_arguments);
OopMap* map = save_live_registers(sasm);
// Called with store_parameter and not C abi
f.load_argument(1, r0); // r0,: array
f.load_argument(0, r1); // r1,: index
int call_offset = __ call_RT(r0, noreg, CAST_FROM_FN_PTR(address, load_flat_array), r0, r1);
oop_maps = new OopMapSet();
oop_maps->add_gc_map(call_offset, map);
restore_live_registers_except_r0(sasm);
// r0: loaded element at array[index]
__ verify_oop(r0);
}
break;
case StubId::c1_store_flat_array_id:
{
StubFrame f(sasm, "store_flat_array", dont_gc_arguments);
OopMap* map = save_live_registers(sasm);
// Called with store_parameter and not C abi
f.load_argument(2, r0); // r0: array
f.load_argument(1, r1); // r1: index
f.load_argument(0, r2); // r2: value
int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, store_flat_array), r0, r1, r2);
oop_maps = new OopMapSet();
oop_maps->add_gc_map(call_offset, map);
restore_live_registers_except_r0(sasm);
}
break;
case StubId::c1_substitutability_check_id:
{
StubFrame f(sasm, "substitutability_check", dont_gc_arguments);
OopMap* map = save_live_registers(sasm);
// Called with store_parameter and not C abi
f.load_argument(1, r1); // r1,: left
f.load_argument(0, r2); // r2,: right
int call_offset = __ call_RT(noreg, noreg, CAST_FROM_FN_PTR(address, substitutability_check), r1, r2);
oop_maps = new OopMapSet();
oop_maps->add_gc_map(call_offset, map);
restore_live_registers_except_r0(sasm);
// r0,: are the two operands substitutable
}
break;
case StubId::c1_register_finalizer_id:
{
__ set_info("register_finalizer", dont_gc_arguments);
@ -824,7 +928,7 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
// load the klass and check the has finalizer flag
Label register_finalizer;
Register t = r5;
__ load_klass(t, r0);
__ load_klass(t, r0, rscratch1);
__ ldrb(t, Address(t, Klass::misc_flags_offset()));
__ tbnz(t, exact_log2(KlassFlags::_misc_has_finalizer), register_finalizer);
__ ret(lr);
@ -851,11 +955,23 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
break;
case StubId::c1_throw_incompatible_class_change_error_id:
{ StubFrame f(sasm, "throw_incompatible_class_cast_exception", dont_gc_arguments, does_not_return);
{ StubFrame f(sasm, "throw_incompatible_class_change_error", dont_gc_arguments, does_not_return);
oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_incompatible_class_change_error), false);
}
break;
case StubId::c1_throw_illegal_monitor_state_exception_id:
{ StubFrame f(sasm, "throw_illegal_monitor_state_exception", dont_gc_arguments);
oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_illegal_monitor_state_exception), false);
}
break;
case StubId::c1_throw_identity_exception_id:
{ StubFrame f(sasm, "throw_identity_exception", dont_gc_arguments);
oop_maps = generate_exception_throw(sasm, CAST_FROM_FN_PTR(address, throw_identity_exception), true);
}
break;
case StubId::c1_slow_subtype_check_id:
{
// Typical calling sequence:
@ -947,7 +1063,7 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
__ br(Assembler::EQ, is_secondary); // Klass is a secondary superclass
// Klass is a concrete class
__ load_klass(r5, obj);
__ load_klass(r5, obj, rscratch1);
__ ldr(rscratch1, Address(r5, r3));
__ cmp(klass, rscratch1);
__ cset(result, Assembler::EQ);
@ -955,7 +1071,7 @@ OopMapSet* Runtime1::generate_code_for(StubId id, StubAssembler* sasm) {
__ bind(is_secondary);
__ load_klass(obj, obj);
__ load_klass(obj, obj, rscratch1);
// This is necessary because I am never in my own secondary_super list.
__ cmp(obj, klass);

3
src/hotspot/cpu/aarch64/c2_CodeStubs_aarch64.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2020, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -59,7 +59,6 @@ void C2EntryBarrierStub::emit(C2_MacroAssembler& masm) {
__ b(continuation());
__ bind(guard());
__ relocate(entry_guard_Relocation::spec());
__ emit_int32(0); // nmethod guard value
}

133
src/hotspot/cpu/aarch64/c2_MacroAssembler_aarch64.cpp
View File

@ -49,6 +49,27 @@
typedef void (MacroAssembler::* chr_insn)(Register Rt, const Address &adr);
void C2_MacroAssembler::entry_barrier() {
BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler();
// Dummy labels for just measuring the code size
Label dummy_slow_path;
Label dummy_continuation;
Label dummy_guard;
Label* slow_path = &dummy_slow_path;
Label* continuation = &dummy_continuation;
Label* guard = &dummy_guard;
if (!Compile::current()->output()->in_scratch_emit_size()) {
// Use real labels from actual stub when not emitting code for the purpose of measuring its size
C2EntryBarrierStub* stub = new (Compile::current()->comp_arena()) C2EntryBarrierStub();
Compile::current()->output()->add_stub(stub);
slow_path = &stub->entry();
continuation = &stub->continuation();
guard = &stub->guard();
}
// In the C2 code, we move the non-hot part of nmethod entry barriers out-of-line to a stub.
bs->nmethod_entry_barrier(this, slow_path, continuation, guard);
}
// jdk.internal.util.ArraysSupport.vectorizedHashCode
address C2_MacroAssembler::arrays_hashcode(Register ary, Register cnt, Register result,
FloatRegister vdata0, FloatRegister vdata1,
@ -167,7 +188,7 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
}
if (DiagnoseSyncOnValueBasedClasses != 0) {
load_klass(t1, obj);
load_klass(t1, obj, rscratch2);
ldrb(t1, Address(t1, Klass::misc_flags_offset()));
tst(t1, KlassFlags::_misc_is_value_based_class);
br(Assembler::NE, slow_path);
@ -204,8 +225,7 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
// Try to lock. Transition lock-bits 0b01 => 0b00
orr(t1_mark, t1_mark, markWord::unlocked_value);
eor(t3_t, t1_mark, markWord::unlocked_value);
cmpxchg(/*addr*/ obj, /*expected*/ t1_mark, /*new*/ t3_t, Assembler::xword,
/*acquire*/ true, /*release*/ false, /*weak*/ false, noreg);
cmpxchg(/*addr*/ obj, /*expected*/ t1_mark, /*new*/ t3_t, Assembler::xword, memory_order_acquire);
br(Assembler::NE, slow_path);
bind(push);
@ -220,6 +240,10 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
bind(inflated);
const Register t1_monitor = t1;
// Offsets into the current thread's object monitor cache (omc).
const ByteSize thr_omc_offset = JavaThread::om_cache_offset();
const ByteSize omc_monitor_offset = OMCache::monitor_offset();
const ByteSize omc_obj_offset = OMCache::obj_offset();
if (!UseObjectMonitorTable) {
assert(t1_monitor == t1_mark, "should be the same here");
@ -230,18 +254,12 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
// Save the mark, we might need it to extract the hash.
mov(t3, t1_mark);
// Look for the monitor in the om_cache.
// Look for the monitor in the current thread's object monitor cache (omc).
ByteSize cache_offset = JavaThread::om_cache_oops_offset();
ByteSize monitor_offset = OMCache::oop_to_monitor_difference();
const int num_unrolled = OMCache::CAPACITY;
for (int i = 0; i < num_unrolled; i++) {
ldr(t1_monitor, Address(rthread, cache_offset + monitor_offset));
ldr(t2, Address(rthread, cache_offset));
cmp(obj, t2);
br(Assembler::EQ, monitor_found);
cache_offset = cache_offset + OMCache::oop_to_oop_difference();
}
ldr(t1_monitor, Address(rthread, thr_omc_offset + omc_monitor_offset));
ldr(t2, Address(rthread, thr_omc_offset + omc_obj_offset));
cmp(obj, t2);
br(Assembler::EQ, monitor_found);
// Look for the monitor in the table.
@ -269,6 +287,10 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
cmp(t3, obj);
br(Assembler::NE, slow_path);
// Store the monitor in the current thread's object monitor cache (omc).
str(t1_monitor, Address(rthread, thr_omc_offset + omc_monitor_offset));
str(obj, Address(rthread, thr_omc_offset + omc_obj_offset));
bind(monitor_found);
}
@ -285,8 +307,7 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
// Try to CAS owner (no owner => current thread's _monitor_owner_id).
ldr(rscratch2, Address(rthread, JavaThread::monitor_owner_id_offset()));
cmpxchg(t2_owner_addr, zr, rscratch2, Assembler::xword, /*acquire*/ true,
/*release*/ false, /*weak*/ false, t3_owner);
cmpxchg(t2_owner_addr, zr, rscratch2, Assembler::xword, memory_order_acquire, t3_owner);
br(Assembler::EQ, monitor_locked);
// Check if recursive.
@ -298,6 +319,7 @@ void C2_MacroAssembler::fast_lock(Register obj, Register box, Register t1,
bind(monitor_locked);
if (UseObjectMonitorTable) {
// Cache the monitor for unlock.
str(t1_monitor, Address(box, BasicLock::object_monitor_cache_offset_in_bytes()));
}
}
@ -371,8 +393,7 @@ void C2_MacroAssembler::fast_unlock(Register obj, Register box, Register t1,
// Try to unlock. Transition lock bits 0b00 => 0b01
assert(oopDesc::mark_offset_in_bytes() == 0, "required to avoid lea");
orr(t3_t, t1_mark, markWord::unlocked_value);
cmpxchg(/*addr*/ obj, /*expected*/ t1_mark, /*new*/ t3_t, Assembler::xword,
/*acquire*/ false, /*release*/ true, /*weak*/ false, noreg);
cmpxchg(/*addr*/ obj, /*expected*/ t1_mark, /*new*/ t3_t, Assembler::xword, memory_order_release);
br(Assembler::EQ, unlocked);
bind(push_and_slow_path);
@ -1509,9 +1530,9 @@ void C2_MacroAssembler::sve_vmask_fromlong(FloatRegister dst, Register src,
// Expected: dst = 0x00 01 01 00 00 01 00 01 01 00 00 00 01 01 00 01
// Put long value from general purpose register into the first lane of vector.
// The higher lanes are set to zero.
// vtmp = 0x0000000000000000 | 0x000000000000658D
sve_dup(vtmp, B, 0);
mov(vtmp, D, 0, src);
fmovd(vtmp, src);
// Transform the value in the first lane which is mask in bit now to the mask in
// byte, which can be done by SVE2's BDEP instruction.
@ -1809,19 +1830,19 @@ void C2_MacroAssembler::neon_reduce_mul_integral(Register dst, BasicType bt,
if (isQ) {
// Multiply the lower half and higher half of vector iteratively.
// vtmp1 = vsrc[8:15]
ins(vtmp1, D, vsrc, 0, 1);
ext(vtmp1, T16B, vsrc, vsrc, 8);
// vtmp1[n] = vsrc[n] * vsrc[n + 8], where n=[0, 7]
mulv(vtmp1, T8B, vtmp1, vsrc);
// vtmp2 = vtmp1[4:7]
ins(vtmp2, S, vtmp1, 0, 1);
ext(vtmp2, T8B, vtmp1, vtmp1, 4);
// vtmp1[n] = vtmp1[n] * vtmp1[n + 4], where n=[0, 3]
mulv(vtmp1, T8B, vtmp2, vtmp1);
} else {
ins(vtmp1, S, vsrc, 0, 1);
ext(vtmp1, T8B, vsrc, vsrc, 4);
mulv(vtmp1, T8B, vtmp1, vsrc);
}
// vtmp2 = vtmp1[2:3]
ins(vtmp2, H, vtmp1, 0, 1);
ext(vtmp2, T8B, vtmp1, vtmp1, 2);
// vtmp2[n] = vtmp1[n] * vtmp1[n + 2], where n=[0, 1]
mulv(vtmp2, T8B, vtmp2, vtmp1);
// dst = vtmp2[0] * isrc * vtmp2[1]
@ -1834,12 +1855,12 @@ void C2_MacroAssembler::neon_reduce_mul_integral(Register dst, BasicType bt,
break;
case T_SHORT:
if (isQ) {
ins(vtmp2, D, vsrc, 0, 1);
ext(vtmp2, T16B, vsrc, vsrc, 8);
mulv(vtmp2, T4H, vtmp2, vsrc);
ins(vtmp1, S, vtmp2, 0, 1);
ext(vtmp1, T8B, vtmp2, vtmp2, 4);
mulv(vtmp1, T4H, vtmp1, vtmp2);
} else {
ins(vtmp1, S, vsrc, 0, 1);
ext(vtmp1, T8B, vsrc, vsrc, 4);
mulv(vtmp1, T4H, vtmp1, vsrc);
}
umov(rscratch1, vtmp1, H, 0);
@ -1851,7 +1872,7 @@ void C2_MacroAssembler::neon_reduce_mul_integral(Register dst, BasicType bt,
break;
case T_INT:
if (isQ) {
ins(vtmp1, D, vsrc, 0, 1);
ext(vtmp1, T16B, vsrc, vsrc, 8);
mulv(vtmp1, T2S, vtmp1, vsrc);
} else {
vtmp1 = vsrc;
@ -1907,19 +1928,19 @@ void C2_MacroAssembler::neon_reduce_mul_fp(FloatRegister dst, BasicType bt,
break;
case T_FLOAT:
fmuls(dst, fsrc, vsrc);
ins(vtmp, S, vsrc, 0, 1);
ext(vtmp, T8B, vsrc, vsrc, 4);
fmuls(dst, dst, vtmp);
if (isQ) {
ins(vtmp, S, vsrc, 0, 2);
ext(vtmp, T16B, vsrc, vsrc, 8);
fmuls(dst, dst, vtmp);
ins(vtmp, S, vsrc, 0, 3);
ext(vtmp, T16B, vsrc, vsrc, 12);
fmuls(dst, dst, vtmp);
}
break;
case T_DOUBLE:
assert(isQ, "unsupported");
fmuld(dst, fsrc, vsrc);
ins(vtmp, D, vsrc, 0, 1);
ext(vtmp, T16B, vsrc, vsrc, 8);
fmuld(dst, dst, vtmp);
break;
default:
@ -2731,7 +2752,8 @@ void C2_MacroAssembler::reconstruct_frame_pointer(Register rtmp) {
void C2_MacroAssembler::select_from_two_vectors_neon(FloatRegister dst, FloatRegister src1,
FloatRegister src2, FloatRegister index,
FloatRegister tmp, unsigned vector_length_in_bytes) {
assert_different_registers(dst, src1, src2, tmp);
assert_different_registers(src2, tmp);
assert_different_registers(index, tmp);
SIMD_Arrangement size = vector_length_in_bytes == 16 ? T16B : T8B;
if (vector_length_in_bytes == 16) {
@ -2760,7 +2782,8 @@ void C2_MacroAssembler::select_from_two_vectors_sve(FloatRegister dst, FloatRegi
FloatRegister src2, FloatRegister index,
FloatRegister tmp, SIMD_RegVariant T,
unsigned vector_length_in_bytes) {
assert_different_registers(dst, src1, src2, index, tmp);
assert_different_registers(src2, tmp);
assert_different_registers(index, tmp);
if (vector_length_in_bytes == 8) {
// We need to fit both the source vectors (src1, src2) in a single vector register because the
@ -2787,7 +2810,8 @@ void C2_MacroAssembler::select_from_two_vectors(FloatRegister dst, FloatRegister
FloatRegister tmp, BasicType bt,
unsigned vector_length_in_bytes) {
assert_different_registers(dst, src1, src2, index, tmp);
assert_different_registers(dst, src1, src2, tmp);
assert_different_registers(index, tmp);
// The cases that can reach this method are -
// - UseSVE = 0/1, vector_length_in_bytes = 8 or 16, excluding double and long types
@ -2970,3 +2994,42 @@ int C2_MacroAssembler::vector_iota_entry_index(BasicType bt) {
ShouldNotReachHere();
}
}
// Vector integer division for BYTE elements. Each BYTE is widened to SHORT for
// the low and high halves of the register, divided using the SHORT helper
// (which widens further to INT), and the two SHORT result halves are narrowed
// back to BYTE.
void C2_MacroAssembler::sve_sdiv_byte(FloatRegister dst_src1, FloatRegister src2,
FloatRegister vtmp1, FloatRegister vtmp2,
FloatRegister vtmp3, FloatRegister vtmp4) {
assert_different_registers(dst_src1, src2, vtmp1, vtmp2, vtmp3, vtmp4);
FloatRegister src1 = dst_src1;
// Low half of the bytes -> SHORT, then divide (result SHORT in vtmp1).
sve_sunpklo(vtmp1, H, src1);
sve_sunpklo(vtmp2, H, src2);
sve_sdiv_short(vtmp1, vtmp2, vtmp3, vtmp4);
// High half of the bytes -> SHORT, then divide (result SHORT in src1).
sve_sunpkhi(src1, H, src1);
sve_sunpkhi(vtmp2, H, src2);
sve_sdiv_short(src1, vtmp2, vtmp3, vtmp4);
// Narrow the two SHORT result halves back to BYTE.
sve_uzp1(dst_src1, B, vtmp1, src1);
}
// Vector integer division for SHORT elements, implemented by widening each
// element to 32 bits, performing SDIV, and narrowing back.
void C2_MacroAssembler::sve_sdiv_short(FloatRegister dst_src1, FloatRegister src2,
FloatRegister vtmp1, FloatRegister vtmp2) {
assert_different_registers(dst_src1, src2, vtmp1, vtmp2);
FloatRegister src1 = dst_src1;
// Low half: SHORT -> INT, then divide.
sve_sunpklo(vtmp1, S, src1);
sve_sunpklo(vtmp2, S, src2);
sve_sdiv(vtmp1, S, ptrue, vtmp2);
// High half: SHORT -> INT, then divide.
sve_sunpkhi(src1, S, src1);
sve_sunpkhi(vtmp2, S, src2);
sve_sdiv(src1, S, ptrue, vtmp2);
// Narrow the two INT result halves back to SHORT.
sve_uzp1(dst_src1, H, vtmp1, src1);
}

7
src/hotspot/cpu/aarch64/c2_MacroAssembler_aarch64.hpp
View File

@ -77,6 +77,8 @@
public:
using Assembler::sve_cpy;
void entry_barrier();
// jdk.internal.util.ArraysSupport.vectorizedHashCode
address arrays_hashcode(Register ary, Register cnt, Register result, FloatRegister vdata0,
FloatRegister vdata1, FloatRegister vdata2, FloatRegister vdata3,
@ -253,4 +255,9 @@
void sve_cpy(FloatRegister dst, SIMD_RegVariant T, PRegister pg, int imm8,
bool isMerge);
int vector_iota_entry_index(BasicType bt);
void sve_sdiv_byte(FloatRegister dst_src1, FloatRegister src2, FloatRegister vtmp1,
FloatRegister vtmp2, FloatRegister vtmp3, FloatRegister vtmp4);
void sve_sdiv_short(FloatRegister dst_src1, FloatRegister src2,
FloatRegister vtmp1, FloatRegister vtmp2);
#endif // CPU_AARCH64_C2_MACROASSEMBLER_AARCH64_HPP

8
src/hotspot/cpu/aarch64/compressedKlass_aarch64.cpp
View File

@ -120,11 +120,7 @@ char* CompressedKlassPointers::reserve_address_space_for_compressed_classes(size
return result;
}
bool CompressedKlassPointers::check_klass_decode_mode(address base, int shift, const size_t range) {
return MacroAssembler::check_klass_decode_mode(base, shift, range);
}
bool CompressedKlassPointers::set_klass_decode_mode() {
void CompressedKlassPointers::initialize_pd() {
const size_t range = klass_range_end() - base();
return MacroAssembler::set_klass_decode_mode(_base, _shift, range);
MacroAssembler::initialize_klass_decode_mode(_base, _shift, range);
}

65
src/hotspot/cpu/aarch64/continuationFreezeThaw_aarch64.inline.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2019, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2019, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -59,23 +59,20 @@ inline frame FreezeBase::sender(const frame& f) {
if (FKind::interpreted) {
return frame(f.sender_sp(), f.interpreter_frame_sender_sp(), f.link(), f.sender_pc());
}
intptr_t** link_addr = link_address<FKind>(f);
intptr_t* sender_sp = (intptr_t*)(link_addr + frame::sender_sp_offset); // f.unextended_sp() + (fsize/wordSize); //
address sender_pc = ContinuationHelper::return_address_at(sender_sp - 1);
assert(sender_sp != f.sp(), "must have changed");
frame::CompiledFramePointers cfp = f.compiled_frame_details();
int slot = 0;
CodeBlob* sender_cb = CodeCache::find_blob_and_oopmap(sender_pc, slot);
CodeBlob* sender_cb = CodeCache::find_blob_and_oopmap(*cfp.sender_pc_addr, slot);
return sender_cb != nullptr
? frame(sender_sp, sender_sp, *link_addr, sender_pc, sender_cb,
slot == -1 ? nullptr : sender_cb->oop_map_for_slot(slot, sender_pc),
false /* on_heap ? */)
: frame(sender_sp, sender_sp, *link_addr, sender_pc);
? frame(cfp.sender_sp, cfp.sender_sp, *cfp.saved_fp_addr, *cfp.sender_pc_addr, sender_cb,
slot == -1 ? nullptr : sender_cb->oop_map_for_slot(slot, *cfp.sender_pc_addr), false)
: frame(cfp.sender_sp, cfp.sender_sp, *cfp.saved_fp_addr, *cfp.sender_pc_addr);
}
template<typename FKind>
frame FreezeBase::new_heap_frame(frame& f, frame& caller) {
frame FreezeBase::new_heap_frame(frame& f, frame& caller, int size_adjust) {
assert(FKind::is_instance(f), "");
assert(!caller.is_interpreted_frame()
|| caller.unextended_sp() == (intptr_t*)caller.at(frame::interpreter_frame_last_sp_offset), "");
@ -109,14 +106,14 @@ frame FreezeBase::new_heap_frame(frame& f, frame& caller) {
fp = FKind::compiled ? *(intptr_t**)(f.sp() - frame::sender_sp_offset) : (intptr_t*)badAddressVal;
int fsize = FKind::size(f);
sp = caller.unextended_sp() - fsize;
if (caller.is_interpreted_frame()) {
sp = caller.unextended_sp() - fsize - size_adjust;
if (caller.is_interpreted_frame() && size_adjust == 0) {
// If the caller is interpreted, our stackargs are not supposed to overlap with it
// so we make more room by moving sp down by argsize
int argsize = FKind::stack_argsize(f);
sp -= argsize;
caller.set_sp(sp + fsize);
}
caller.set_sp(sp + fsize);
assert(_cont.tail()->is_in_chunk(sp), "");
@ -183,11 +180,12 @@ inline void FreezeBase::set_top_frame_metadata_pd(const frame& hf) {
: (intptr_t)hf.fp();
}
inline void FreezeBase::patch_pd(frame& hf, const frame& caller) {
inline void FreezeBase::patch_pd(frame& hf, const frame& caller, bool is_bottom_frame) {
if (caller.is_interpreted_frame()) {
assert(!caller.is_empty(), "");
patch_callee_link_relative(caller, caller.fp());
} else {
} else if (is_bottom_frame && caller.pc() != nullptr) {
assert(caller.is_compiled_frame(), "");
// If we're the bottom-most frame frozen in this freeze, the caller might have stayed frozen in the chunk,
// and its oop-containing fp fixed. We've now just overwritten it, so we must patch it back to its value
// as read from the chunk.
@ -215,7 +213,8 @@ inline intptr_t* AnchorMark::anchor_mark_set_pd() {
if (sp != _last_sp_from_frame) {
// We need to move up return pc and fp. They will be read next in
// set_anchor() and set as _last_Java_pc and _last_Java_fp respectively.
_last_sp_from_frame[-1] = (intptr_t)_top_frame.pc();
ContinuationHelper::patch_return_address_at(&_last_sp_from_frame[-1],
_top_frame.pc());
_last_sp_from_frame[-2] = (intptr_t)_top_frame.fp();
}
_is_interpreted = true;
@ -230,7 +229,7 @@ inline void AnchorMark::anchor_mark_clear_pd() {
_top_frame.interpreter_frame_set_last_sp(_last_sp_from_frame);
intptr_t* sp = _top_frame.sp();
if (sp != _last_sp_from_frame) {
sp[-1] = (intptr_t)_top_frame.pc();
ContinuationHelper::patch_return_address_at(&sp[-1], _top_frame.pc());
}
}
}
@ -258,7 +257,7 @@ inline frame ThawBase::new_entry_frame() {
return frame(sp, sp, _cont.entryFP(), _cont.entryPC()); // TODO PERF: This finds code blob and computes deopt state
}
template<typename FKind> frame ThawBase::new_stack_frame(const frame& hf, frame& caller, bool bottom) {
template<typename FKind> frame ThawBase::new_stack_frame(const frame& hf, frame& caller, bool bottom, int size_adjust) {
assert(FKind::is_instance(hf), "");
// The values in the returned frame object will be written into the callee's stack in patch.
@ -286,24 +285,23 @@ template<typename FKind> frame ThawBase::new_stack_frame(const frame& hf, frame&
return f;
} else {
int fsize = FKind::size(hf);
intptr_t* frame_sp = caller.unextended_sp() - fsize;
intptr_t* frame_sp = caller.unextended_sp() - fsize - size_adjust;
if (bottom || caller.is_interpreted_frame()) {
int argsize = FKind::stack_argsize(hf);
fsize += argsize;
frame_sp -= argsize;
caller.set_sp(caller.sp() - argsize);
assert(caller.sp() == frame_sp + (fsize-argsize), "");
if (size_adjust == 0) {
int argsize = FKind::stack_argsize(hf);
frame_sp -= argsize;
}
frame_sp = align(hf, frame_sp, caller, bottom);
caller.set_sp(frame_sp + fsize + size_adjust);
}
assert(is_aligned(frame_sp, frame::frame_alignment), "");
assert(hf.cb() != nullptr, "");
assert(hf.oop_map() != nullptr, "");
intptr_t* fp;
if (PreserveFramePointer) {
// we need to recreate a "real" frame pointer, pointing into the stack
fp = frame_sp + FKind::size(hf) - frame::sender_sp_offset;
fp = frame_sp + fsize - frame::sender_sp_offset;
} else {
fp = FKind::stub || FKind::native
? frame_sp + fsize - frame::sender_sp_offset // fp always points to the address below the pushed return pc. We need correct address.
@ -318,7 +316,6 @@ inline intptr_t* ThawBase::align(const frame& hf, intptr_t* frame_sp, frame& cal
if (((intptr_t)frame_sp & 0xf) != 0) {
assert(caller.is_interpreted_frame() || (bottom && hf.compiled_frame_stack_argsize() % 2 != 0), "");
frame_sp--;
caller.set_sp(caller.sp() - 1);
}
assert(is_aligned(frame_sp, frame::frame_alignment), "");
#endif
@ -327,7 +324,9 @@ inline intptr_t* ThawBase::align(const frame& hf, intptr_t* frame_sp, frame& cal
}
inline void ThawBase::patch_pd(frame& f, const frame& caller) {
patch_callee_link(caller, caller.fp());
if (caller.is_interpreted_frame() || PreserveFramePointer) {
patch_callee_link(caller, caller.fp());
}
}
inline void ThawBase::patch_pd(frame& f, intptr_t* caller_sp) {
@ -340,7 +339,8 @@ inline intptr_t* ThawBase::push_cleanup_continuation() {
intptr_t* sp = enterSpecial.sp();
// We only need to set the return pc. rfp will be restored back in gen_continuation_enter().
sp[-1] = (intptr_t)ContinuationEntry::cleanup_pc();
ContinuationHelper::patch_return_address_at(&sp[-1],
ContinuationEntry::cleanup_pc());
return sp;
}
@ -349,7 +349,8 @@ inline intptr_t* ThawBase::push_preempt_adapter() {
intptr_t* sp = enterSpecial.sp();
// We only need to set the return pc. rfp will be restored back in generate_cont_preempt_stub().
sp[-1] = (intptr_t)StubRoutines::cont_preempt_stub();
ContinuationHelper::patch_return_address_at(&sp[-1],
StubRoutines::cont_preempt_stub());
return sp;
}

8
src/hotspot/cpu/aarch64/continuationHelper_aarch64.inline.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2022, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2022, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -47,7 +47,8 @@ static inline void patch_return_pc_with_preempt_stub(frame& f) {
// Instead, we will patch the return from the runtime stub back to the
// compiled method so that the target returns to the preempt cleanup stub.
intptr_t* caller_sp = f.sp() + f.cb()->frame_size();
caller_sp[-1] = (intptr_t)StubRoutines::cont_preempt_stub();
ContinuationHelper::patch_return_address_at(&caller_sp[-1],
StubRoutines::cont_preempt_stub());
} else {
// The target will check for preemption once it returns to the interpreter
// or the native wrapper code and will manually jump to the preempt stub.
@ -124,7 +125,8 @@ inline intptr_t** ContinuationHelper::Frame::callee_link_address(const frame& f)
}
inline address* ContinuationHelper::Frame::return_pc_address(const frame& f) {
return (address*)(f.real_fp() - 1);
frame::CompiledFramePointers cfp = f.compiled_frame_details();
return cfp.sender_pc_addr;
}
inline address* ContinuationHelper::InterpretedFrame::return_pc_address(const frame& f) {

6
src/hotspot/cpu/aarch64/foreignGlobals_aarch64.cpp
View File

@ -50,15 +50,15 @@ const ABIDescriptor ForeignGlobals::parse_abi_descriptor(jobject jabi) {
oop abi_oop = JNIHandles::resolve_non_null(jabi);
ABIDescriptor abi;
objArrayOop inputStorage = jdk_internal_foreign_abi_ABIDescriptor::inputStorage(abi_oop);
refArrayOop inputStorage = jdk_internal_foreign_abi_ABIDescriptor::inputStorage(abi_oop);
parse_register_array(inputStorage, StorageType::INTEGER, abi._integer_argument_registers, as_Register);
parse_register_array(inputStorage, StorageType::VECTOR, abi._vector_argument_registers, as_FloatRegister);
objArrayOop outputStorage = jdk_internal_foreign_abi_ABIDescriptor::outputStorage(abi_oop);
refArrayOop outputStorage = jdk_internal_foreign_abi_ABIDescriptor::outputStorage(abi_oop);
parse_register_array(outputStorage, StorageType::INTEGER, abi._integer_return_registers, as_Register);
parse_register_array(outputStorage, StorageType::VECTOR, abi._vector_return_registers, as_FloatRegister);
objArrayOop volatileStorage = jdk_internal_foreign_abi_ABIDescriptor::volatileStorage(abi_oop);
refArrayOop volatileStorage = jdk_internal_foreign_abi_ABIDescriptor::volatileStorage(abi_oop);
parse_register_array(volatileStorage, StorageType::INTEGER, abi._integer_additional_volatile_registers, as_Register);
parse_register_array(volatileStorage, StorageType::VECTOR, abi._vector_additional_volatile_registers, as_FloatRegister);

96
src/hotspot/cpu/aarch64/frame_aarch64.cpp
View File

@ -153,14 +153,17 @@ bool frame::safe_for_sender(JavaThread *thread) {
if (!thread->is_in_full_stack_checked((address)sender_sp)) {
return false;
}
sender_unextended_sp = sender_sp;
// Note: frame::sender_sp_offset is only valid for compiled frame
saved_fp = (intptr_t*) *(sender_sp - frame::sender_sp_offset);
intptr_t **saved_fp_addr = (intptr_t**) (sender_sp - frame::sender_sp_offset);
saved_fp = *saved_fp_addr;
// Note: PAC authentication may fail in case broken frame is passed in.
// Just strip it for now.
sender_pc = pauth_strip_pointer((address) *(sender_sp - 1));
}
// Repair the sender sp if this is a method with scalarized inline type args
sender_sp = repair_sender_sp(sender_sp, saved_fp_addr);
sender_unextended_sp = sender_sp;
}
if (Continuation::is_return_barrier_entry(sender_pc)) {
// sender_pc might be invalid so check that the frame
// actually belongs to a Continuation.
@ -622,13 +625,24 @@ void frame::describe_pd(FrameValues& values, int frame_no) {
ret_pc_loc = fp() + return_addr_offset;
fp_loc = fp();
} else {
ret_pc_loc = real_fp() - return_addr_offset;
fp_loc = real_fp() - sender_sp_offset;
if (cb()->is_nmethod() && cb()->as_nmethod()->needs_stack_repair()) {
values.describe(frame_no, real_fp() - sender_sp_offset - 1, err_msg("fsize for #%d", frame_no), 1);
}
frame::CompiledFramePointers cfp = compiled_frame_details();
ret_pc_loc = (intptr_t*)cfp.sender_pc_addr;
fp_loc = (intptr_t*)cfp.saved_fp_addr;
}
address ret_pc = *(address*)ret_pc_loc;
values.describe(frame_no, ret_pc_loc,
Continuation::is_return_barrier_entry(ret_pc) ? "return address (return barrier)" : "return address");
values.describe(-1, fp_loc, "saved fp", 0); // "unowned" as value belongs to sender
intptr_t* ret_pc_loc2 = real_fp() - return_addr_offset;
if (ret_pc_loc2 != ret_pc_loc) {
intptr_t* fp_loc2 = real_fp() - sender_sp_offset;
values.describe(frame_no, ret_pc_loc2, "return address copy #2");
values.describe(-1, fp_loc2, "saved fp copy #2", 0);
}
}
}
#endif
@ -779,6 +793,78 @@ frame::frame(void* sp, void* fp, void* pc) {
#endif
// Check for a method with scalarized inline type arguments that needs
// a stack repair and return the repaired sender stack pointer.
intptr_t* frame::repair_sender_sp(intptr_t* sender_sp, intptr_t** saved_fp_addr) const {
nmethod* nm = _cb->as_nmethod_or_null();
if (nm != nullptr && nm->needs_stack_repair()) {
// The stack increment resides just below the saved FP on the stack and
// records the total frame size excluding the two words for saving FP and LR
// (see MacroAssembler::remove_frame).
intptr_t* sp_inc_addr = (intptr_t*) (saved_fp_addr - 1);
assert(*sp_inc_addr % StackAlignmentInBytes == 0, "sp_inc not aligned");
int real_frame_size = (*sp_inc_addr / wordSize) + metadata_words_at_bottom;
assert(real_frame_size >= _cb->frame_size() && real_frame_size <= 1000000, "invalid frame size");
sender_sp = unextended_sp() + real_frame_size;
}
return sender_sp;
}
// See comment in MacroAssembler::remove_frame
frame::CompiledFramePointers frame::compiled_frame_details() const {
// we cannot rely upon the last fp having been saved to the thread
// in C2 code but it will have been pushed onto the stack. so we
// have to find it relative to the unextended sp
assert(_cb->frame_size() > 0, "must have non-zero frame size");
// if need stack repair: the bottom of the fake frame, under LR #2
// else the bottom of the frame
intptr_t* l_sender_sp = (!PreserveFramePointer || _sp_is_trusted)
? unextended_sp() + _cb->frame_size()
: sender_sp();
assert(!_sp_is_trusted || l_sender_sp == real_fp(), "");
// the actual bottom of the frame. This actually changes something if the frame needs stack repair
l_sender_sp = repair_sender_sp(l_sender_sp, (intptr_t**)(l_sender_sp - frame::sender_sp_offset));
// From the sender's sp, we can locate the real saved lr (x30) and rfp (x29): they are
// immediately above, no matter if the stack was extended or not
CompiledFramePointers cfp;
cfp.sender_sp = l_sender_sp;
cfp.saved_fp_addr = (intptr_t**)(l_sender_sp - frame::sender_sp_offset);
cfp.sender_pc_addr = (address*)(l_sender_sp - frame::return_addr_offset);
return cfp;
}
intptr_t* frame::repair_sender_sp(nmethod* nm, intptr_t* sp, intptr_t** saved_fp_addr) {
assert(nm != nullptr && nm->needs_stack_repair(), "");
// The stack increment resides just below the saved FP on the stack and
// records the total frame size excluding the two words for saving FP and LR
// (see MacroAssembler::remove_frame).
intptr_t* real_frame_size_addr = (intptr_t*) (saved_fp_addr - 1);
int real_frame_size = (*real_frame_size_addr / wordSize) + metadata_words_at_bottom;
assert(real_frame_size >= nm->frame_size() && real_frame_size <= 1000000, "invalid frame size");
return sp + real_frame_size;
}
bool frame::was_augmented_on_entry(int& real_size) const {
assert(is_compiled_frame(), "");
if (_cb->as_nmethod_or_null()->needs_stack_repair()) {
// The stack increment resides just below the saved FP on the stack and
// records the total frame size excluding the two words for saving FP and LR
// (see MacroAssembler::remove_frame).
intptr_t* real_frame_size_addr = unextended_sp() + _cb->frame_size() - sender_sp_offset - 1;
log_trace(continuations)("real_frame_size is addr is " INTPTR_FORMAT, p2i(real_frame_size_addr));
real_size = (*real_frame_size_addr / wordSize) + metadata_words_at_bottom;
return real_size != _cb->frame_size();
}
real_size = _cb->frame_size();
return false;
}
void JavaFrameAnchor::make_walkable() {
// last frame set?
if (last_Java_sp() == nullptr) return;

16
src/hotspot/cpu/aarch64/frame_aarch64.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 1997, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 1997, 2026, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2014, Red Hat Inc. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -66,7 +66,6 @@
public:
enum {
pc_return_offset = 0,
// All frames
link_offset = 0,
return_addr_offset = 1,
@ -151,6 +150,17 @@
}
public:
// Support for scalarized inline type calling convention
intptr_t* repair_sender_sp(intptr_t* sender_sp, intptr_t** saved_fp_addr) const;
struct CompiledFramePointers {
intptr_t* sender_sp; // The top of the stack of the sender
intptr_t** saved_fp_addr; // Where rfp (x29) is saved on the stack (FP #1 in remove_frame's comment)
address* sender_pc_addr; // Where lr (x30) is saved on the stack (LR #1)
};
CompiledFramePointers compiled_frame_details() const;
static intptr_t* repair_sender_sp(nmethod* nm, intptr_t* sp, intptr_t** saved_fp_addr);
bool was_augmented_on_entry(int& real_size) const;
// Constructors
frame(intptr_t* sp, intptr_t* fp, address pc);
@ -187,8 +197,6 @@
// deoptimization support
void interpreter_frame_set_last_sp(intptr_t* sp);
static jint interpreter_frame_expression_stack_direction() { return -1; }
// returns the sending frame, without applying any barriers
inline frame sender_raw(RegisterMap* map) const;

40
src/hotspot/cpu/aarch64/frame_aarch64.inline.hpp
View File

@ -32,6 +32,9 @@
#include "interpreter/interpreter.hpp"
#include "runtime/sharedRuntime.hpp"
#include "pauth_aarch64.hpp"
#ifdef COMPILER1
#include "c1/c1_Runtime1.hpp"
#endif
// Inline functions for AArch64 frames:
@ -444,28 +447,31 @@ inline frame frame::sender_raw(RegisterMap* map) const {
}
inline frame frame::sender_for_compiled_frame(RegisterMap* map) const {
// we cannot rely upon the last fp having been saved to the thread
// in C2 code but it will have been pushed onto the stack. so we
// have to find it relative to the unextended sp
assert(_cb->frame_size() > 0, "must have non-zero frame size");
intptr_t* l_sender_sp = (!PreserveFramePointer || _sp_is_trusted) ? unextended_sp() + _cb->frame_size()
: sender_sp();
assert(!_sp_is_trusted || l_sender_sp == real_fp(), "");
CompiledFramePointers cfp = compiled_frame_details();
// The return_address is always the word on the stack.
// For ROP protection, C1/C2 will have signed the sender_pc,
// but there is no requirement to authenticate it here.
address sender_pc = pauth_strip_verifiable((address) *(l_sender_sp - 1));
intptr_t** saved_fp_addr = (intptr_t**) (l_sender_sp - frame::sender_sp_offset);
address sender_pc = pauth_strip_verifiable(*cfp.sender_pc_addr);
if (map->update_map()) {
// Tell GC to use argument oopmaps for some runtime stubs that need it.
// For C1, the runtime stub might not have oop maps, so set this flag
// outside of update_register_map.
if (!_cb->is_nmethod()) { // compiled frames do not use callee-saved registers
map->set_include_argument_oops(_cb->caller_must_gc_arguments(map->thread()));
bool c1_buffering = false;
#ifdef COMPILER1
nmethod* nm = _cb->as_nmethod_or_null();
if (nm != nullptr && nm->is_compiled_by_c1() && nm->method()->has_scalarized_args() &&
pc() < nm->verified_inline_entry_point()) {
// The VEP and VIEP(RO) of C1-compiled methods call buffer_inline_args_xxx
// before doing any argument shuffling, so we need to scan the oops
// as the caller passes them.
c1_buffering = true;
}
#endif
if (!_cb->is_nmethod() || c1_buffering) { // compiled frames do not use callee-saved registers
bool caller_args = _cb->caller_must_gc_arguments(map->thread()) || c1_buffering;
map->set_include_argument_oops(caller_args);
if (oop_map() != nullptr) {
_oop_map->update_register_map(this, map);
}
@ -478,19 +484,19 @@ inline frame frame::sender_for_compiled_frame(RegisterMap* map) const {
// Since the prolog does the save and restore of FP there is no oopmap
// for it so we must fill in its location as if there was an oopmap entry
// since if our caller was compiled code there could be live jvm state in it.
update_map_with_saved_link(map, saved_fp_addr);
update_map_with_saved_link(map, cfp.saved_fp_addr);
}
if (Continuation::is_return_barrier_entry(sender_pc)) {
if (map->walk_cont()) { // about to walk into an h-stack
return Continuation::top_frame(*this, map);
} else {
return Continuation::continuation_bottom_sender(map->thread(), *this, l_sender_sp);
return Continuation::continuation_bottom_sender(map->thread(), *this, cfp.sender_sp);
}
}
intptr_t* unextended_sp = l_sender_sp;
return frame(l_sender_sp, unextended_sp, *saved_fp_addr, sender_pc);
intptr_t* unextended_sp = cfp.sender_sp;
return frame(cfp.sender_sp, unextended_sp, *cfp.saved_fp_addr, sender_pc);
}
template <typename RegisterMapT>

55
src/hotspot/cpu/aarch64/gc/g1/g1BarrierSetAssembler_aarch64.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2018, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2018, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -132,6 +132,7 @@ void G1BarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* mas
static void generate_queue_test_and_insertion(MacroAssembler* masm, ByteSize index_offset, ByteSize buffer_offset, Label& runtime,
const Register thread, const Register value, const Register temp1, const Register temp2) {
assert_different_registers(value, temp1, temp2);
// Can we store a value in the given thread's buffer?
// (The index field is typed as size_t.)
__ ldr(temp1, Address(thread, in_bytes(index_offset))); // temp1 := *(index address)
@ -205,6 +206,7 @@ void G1BarrierSetAssembler::g1_write_barrier_pre(MacroAssembler* masm,
__ bind(runtime);
assert_different_registers(rscratch1, pre_val); // push_call_clobbered_registers trashes rscratch1
__ push_call_clobbered_registers();
// Calling the runtime using the regular call_VM_leaf mechanism generates
@ -383,6 +385,16 @@ void G1BarrierSetAssembler::load_at(MacroAssembler* masm, DecoratorSet decorator
void G1BarrierSetAssembler::oop_store_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
Address dst, Register val, Register tmp1, Register tmp2, Register tmp3) {
bool in_heap = (decorators & IN_HEAP) != 0;
bool as_normal = (decorators & AS_NORMAL) != 0;
bool dest_uninitialized = (decorators & IS_DEST_UNINITIALIZED) != 0;
bool needs_pre_barrier = as_normal && !dest_uninitialized;
bool needs_post_barrier = (val != noreg && in_heap);
assert_different_registers(val, tmp1, tmp2, tmp3);
// flatten object address if needed
if (dst.index() == noreg && dst.offset() == 0) {
if (dst.base() != tmp3) {
@ -392,31 +404,38 @@ void G1BarrierSetAssembler::oop_store_at(MacroAssembler* masm, DecoratorSet deco
__ lea(tmp3, dst);
}
g1_write_barrier_pre(masm,
tmp3 /* obj */,
tmp2 /* pre_val */,
rthread /* thread */,
tmp1 /* tmp1 */,
rscratch2 /* tmp2 */,
val != noreg /* tosca_live */,
false /* expand_call */);
if (needs_pre_barrier) {
g1_write_barrier_pre(masm,
tmp3 /* obj */,
tmp2 /* pre_val */,
rthread /* thread */,
tmp1 /* tmp1 */,
rscratch2 /* tmp2 */,
val != noreg /* tosca_live */,
false /* expand_call */);
}
if (val == noreg) {
BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp3, 0), noreg, noreg, noreg, noreg);
} else {
// G1 barrier needs uncompressed oop for region cross check.
Register new_val = val;
if (UseCompressedOops) {
new_val = rscratch2;
__ mov(new_val, val);
if (needs_post_barrier) {
if (UseCompressedOops) {
new_val = rscratch2;
__ mov(new_val, val);
}
}
BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp3, 0), val, noreg, noreg, noreg);
g1_write_barrier_post(masm,
tmp3 /* store_adr */,
new_val /* new_val */,
rthread /* thread */,
tmp1 /* tmp1 */,
tmp2 /* tmp2 */);
if (needs_post_barrier) {
g1_write_barrier_post(masm,
tmp3 /* store_adr */,
new_val /* new_val */,
rthread /* thread */,
tmp1 /* tmp1 */,
tmp2 /* tmp2 */);
}
}
}

137
src/hotspot/cpu/aarch64/gc/g1/g1_aarch64.ad
View File

@ -1,5 +1,5 @@
//
// Copyright (c) 2024, 2025, Oracle and/or its affiliates. All rights reserved.
// Copyright (c) 2024, 2026, Oracle and/or its affiliates. All rights reserved.
// DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
//
// This code is free software; you can redistribute it and/or modify it
@ -73,6 +73,121 @@ static void write_barrier_post(MacroAssembler* masm,
%}
// TODO 8350865 (same applies to g1StoreLSpecialTwoOops)
// - Do not set/overwrite barrier data here, also handle G1C2BarrierPostNotNull
// - Move this into the .m4?
instruct g1StoreLSpecialOneOopOff0(indirect mem, iRegLNoSp src, immI0 off, iRegPNoSp tmp1, iRegPNoSp tmp2, iRegPNoSp tmp3, rFlagsReg cr)
%{
predicate(UseG1GC);
match(Set mem (StoreLSpecial mem (Binary src off)));
effect(TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr);
ins_cost(INSN_COST);
format %{ "str $src, $mem\t# g1StoreLSpecialOneOopOff0" %}
ins_encode %{
((MachNode*)this)->set_barrier_data(G1C2BarrierPre | G1C2BarrierPost);
write_barrier_pre(masm, this,
$mem$$Register /* obj */,
$tmp1$$Register /* pre_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $src$$Register) /* preserve */);
__ str($src$$Register, $mem$$Register);
// Extract the narrow oop field value
__ ubfm($tmp1$$Register, $src$$Register, 0, 31);
__ decode_heap_oop($tmp1$$Register, $tmp1$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
$tmp1$$Register /* new_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */);
%}
ins_pipe(istore_reg_mem);
%}
instruct g1StoreLSpecialOneOopOff4(indirect mem, iRegLNoSp src, immI_4 off, iRegPNoSp tmp1, iRegPNoSp tmp2, iRegPNoSp tmp3, iRegPNoSp tmp4, rFlagsReg cr)
%{
predicate(UseG1GC);
match(Set mem (StoreLSpecial mem (Binary src off)));
effect(TEMP tmp1, TEMP tmp2, TEMP tmp3, TEMP tmp4, KILL cr);
ins_cost(INSN_COST);
format %{ "str $src, $mem\t# g1StoreLSpecialOneOopOff4" %}
ins_encode %{
((MachNode*)this)->set_barrier_data(G1C2BarrierPre | G1C2BarrierPost);
// Adjust address to point to narrow oop
__ add($tmp4$$Register, $mem$$Register, 4);
write_barrier_pre(masm, this,
$tmp4$$Register /* obj */,
$tmp1$$Register /* pre_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $src$$Register, $tmp4$$Register) /* preserve */);
__ str($src$$Register, $mem$$Register);
// Shift long value to extract the narrow oop field value
__ lsr($tmp1$$Register, $src$$Register, 32);
__ decode_heap_oop($tmp1$$Register, $tmp1$$Register);
write_barrier_post(masm, this,
$tmp4$$Register /* store_addr */,
$tmp1$$Register /* new_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */);
%}
ins_pipe(istore_reg_mem);
%}
instruct g1StoreLSpecialTwoOops(indirect mem, iRegLNoSp src, iRegPNoSp tmp1, iRegPNoSp tmp2, iRegPNoSp tmp3, iRegPNoSp tmp4, rFlagsReg cr)
%{
predicate(UseG1GC);
match(Set mem (StoreLSpecial mem src));
effect(TEMP tmp1, TEMP tmp2, TEMP tmp3, TEMP tmp4, KILL cr);
ins_cost(INSN_COST);
format %{ "str $src, $mem\t# g1StoreLSpecialTwoOops" %}
ins_encode %{
((MachNode*)this)->set_barrier_data(G1C2BarrierPre | G1C2BarrierPost);
write_barrier_pre(masm, this,
$mem$$Register /* obj */,
$tmp1$$Register /* pre_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $src$$Register) /* preserve */);
// Adjust address to point to the second narrow oop in the long value
__ add($tmp4$$Register, $mem$$Register, 4);
write_barrier_pre(masm, this,
$tmp4$$Register /* obj */,
$tmp1$$Register /* pre_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $src$$Register, $tmp4$$Register) /* preserve */);
__ str($src$$Register, $mem$$Register);
// Zero-extend first narrow oop to long
__ ubfm($tmp1$$Register, $src$$Register, 0, 31);
__ decode_heap_oop($tmp1$$Register, $tmp1$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
$tmp1$$Register /* new_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */);
// Shift long value to extract the second narrow oop field value
__ lsr($tmp1$$Register, $src$$Register, 32);
__ decode_heap_oop($tmp1$$Register, $tmp1$$Register);
write_barrier_post(masm, this,
$tmp4$$Register /* store_addr */,
$tmp1$$Register /* new_val */,
$tmp2$$Register /* tmp1 */,
$tmp3$$Register /* tmp2 */);
%}
ins_pipe(istore_reg_mem);
%}
// BEGIN This section of the file is automatically generated. Do not edit --------------
// This section is generated from g1_aarch64.m4
@ -283,7 +398,7 @@ instruct g1CompareAndExchangeP(iRegPNoSp res, indirect mem, iRegP oldval, iRegP
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
false /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_release, $res$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
$newval$$Register /* new_val */,
@ -316,7 +431,7 @@ instruct g1CompareAndExchangePAcq(iRegPNoSp res, indirect mem, iRegP oldval, iRe
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
true /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_seq_cst, $res$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
$newval$$Register /* new_val */,
@ -346,7 +461,7 @@ instruct g1CompareAndExchangeN(iRegNNoSp res, indirect mem, iRegN oldval, iRegN
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
false /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_release, $res$$Register);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -377,7 +492,7 @@ instruct g1CompareAndExchangeNAcq(iRegNNoSp res, indirect mem, iRegN oldval, iRe
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
true /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_seq_cst, $res$$Register);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -409,8 +524,7 @@ instruct g1CompareAndSwapP(iRegINoSp res, indirect mem, iRegP newval, iRegPNoSp
$tmp2$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
false /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_release);
__ cset($res$$Register, Assembler::EQ);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -442,8 +556,7 @@ instruct g1CompareAndSwapPAcq(iRegINoSp res, indirect mem, iRegP newval, iRegPNo
$tmp2$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
true /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, memory_order_seq_cst);
__ cset($res$$Register, Assembler::EQ);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -475,8 +588,7 @@ instruct g1CompareAndSwapN(iRegINoSp res, indirect mem, iRegN newval, iRegPNoSp
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
false /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_release);
__ cset($res$$Register, Assembler::EQ);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,
@ -509,8 +621,7 @@ instruct g1CompareAndSwapNAcq(iRegINoSp res, indirect mem, iRegN newval, iRegPNo
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
true /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, memory_order_seq_cst);
__ cset($res$$Register, Assembler::EQ);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,

26
src/hotspot/cpu/aarch64/gc/g1/g1_aarch64.m4
View File

@ -151,7 +151,7 @@ instruct g1CompareAndExchangeP$1(iRegPNoSp res, indirect mem, iRegP oldval, iReg
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
$3 /* acquire */, true /* release */, false /* weak */, $res$$Register);
ifelse($1,Acq,memory_order_seq_cst,memory_order_release), $res$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
$newval$$Register /* new_val */,
@ -160,8 +160,8 @@ instruct g1CompareAndExchangeP$1(iRegPNoSp res, indirect mem, iRegP oldval, iReg
%}
ins_pipe(pipe_slow);
%}')dnl
CAEP_INSN(,,false)
CAEP_INSN(Acq,_acq,true)
CAEP_INSN(,)
CAEP_INSN(Acq,_acq)
dnl
define(`CAEN_INSN',
`
@ -185,7 +185,7 @@ instruct g1CompareAndExchangeN$1(iRegNNoSp res, indirect mem, iRegN oldval, iReg
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
$3 /* acquire */, true /* release */, false /* weak */, $res$$Register);
ifelse($1,Acq,memory_order_seq_cst,memory_order_release), $res$$Register);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -195,8 +195,8 @@ instruct g1CompareAndExchangeN$1(iRegNNoSp res, indirect mem, iRegN oldval, iReg
%}
ins_pipe(pipe_slow);
%}')dnl
CAEN_INSN(,,false)
CAEN_INSN(Acq,_acq,true)
CAEN_INSN(,)
CAEN_INSN(Acq,_acq)
dnl
define(`CASP_INSN',
`
@ -221,8 +221,7 @@ instruct g1CompareAndSwapP$1(iRegINoSp res, indirect mem, iRegP newval, iRegPNoS
$tmp2$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword,
$3 /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::xword, ifelse($1,Acq,memory_order_seq_cst,memory_order_release));
__ cset($res$$Register, Assembler::EQ);
write_barrier_post(masm, this,
$mem$$Register /* store_addr */,
@ -232,8 +231,8 @@ instruct g1CompareAndSwapP$1(iRegINoSp res, indirect mem, iRegP newval, iRegPNoS
%}
ins_pipe(pipe_slow);
%}')dnl
CASP_INSN(,,false)
CASP_INSN(Acq,_acq,true)
CASP_INSN(,)
CASP_INSN(Acq,_acq)
dnl
define(`CASN_INSN',
`
@ -258,8 +257,7 @@ instruct g1CompareAndSwapN$1(iRegINoSp res, indirect mem, iRegN newval, iRegPNoS
$tmp3$$Register /* tmp2 */,
RegSet::of($mem$$Register, $oldval$$Register, $newval$$Register) /* preserve */,
RegSet::of($res$$Register) /* no_preserve */);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word,
$3 /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval$$Register, $newval$$Register, Assembler::word, ifelse($1,Acq,memory_order_seq_cst,memory_order_release));
__ cset($res$$Register, Assembler::EQ);
__ decode_heap_oop($tmp1$$Register, $newval$$Register);
write_barrier_post(masm, this,
@ -270,8 +268,8 @@ instruct g1CompareAndSwapN$1(iRegINoSp res, indirect mem, iRegN newval, iRegPNoS
%}
ins_pipe(pipe_slow);
%}')dnl
CASN_INSN(,,false)
CASN_INSN(Acq,_acq,true)
CASN_INSN(,)
CASN_INSN(Acq,_acq)
dnl
define(`XCHGP_INSN',
`

45
src/hotspot/cpu/aarch64/gc/shared/barrierSetAssembler_aarch64.cpp
View File

@ -26,6 +26,7 @@
#include "gc/shared/barrierSet.hpp"
#include "gc/shared/barrierSetAssembler.hpp"
#include "gc/shared/barrierSetNMethod.hpp"
#include "gc/shared/barrierSetRuntime.hpp"
#include "gc/shared/collectedHeap.hpp"
#include "interpreter/interp_masm.hpp"
#include "memory/universe.hpp"
@ -86,22 +87,35 @@ void BarrierSetAssembler::store_at(MacroAssembler* masm, DecoratorSet decorators
Address dst, Register val, Register tmp1, Register tmp2, Register tmp3) {
bool in_heap = (decorators & IN_HEAP) != 0;
bool in_native = (decorators & IN_NATIVE) != 0;
bool is_not_null = (decorators & IS_NOT_NULL) != 0;
switch (type) {
case T_OBJECT:
case T_ARRAY: {
val = val == noreg ? zr : val;
if (in_heap) {
if (UseCompressedOops) {
assert(!dst.uses(val), "not enough registers");
if (val != zr) {
__ encode_heap_oop(val);
if (val == noreg) {
assert(!is_not_null, "inconsistent access");
if (UseCompressedOops) {
__ strw(zr, dst);
} else {
__ str(zr, dst);
}
__ strw(val, dst);
} else {
__ str(val, dst);
if (UseCompressedOops) {
assert(!dst.uses(val), "not enough registers");
if (is_not_null) {
__ encode_heap_oop_not_null(val);
} else {
__ encode_heap_oop(val);
}
__ strw(val, dst);
} else {
__ str(val, dst);
}
}
} else {
assert(in_native, "why else?");
assert(val != noreg, "not supported");
__ str(val, dst);
}
break;
@ -122,6 +136,19 @@ void BarrierSetAssembler::store_at(MacroAssembler* masm, DecoratorSet decorators
}
}
void BarrierSetAssembler::flat_field_copy(MacroAssembler* masm, DecoratorSet decorators,
Register src, Register dst, Register inline_layout_info) {
// flat_field_copy implementation is fairly complex, and there are not any
// "short-cuts" to be made from asm. What there is, appears to have the same
// cost in C++, so just "call_VM_leaf" for now rather than maintain hundreds
// of hand-rolled instructions...
if (decorators & IS_DEST_UNINITIALIZED) {
__ call_VM_leaf(CAST_FROM_FN_PTR(address, BarrierSetRuntime::value_copy_is_dest_uninitialized), src, dst, inline_layout_info);
} else {
__ call_VM_leaf(CAST_FROM_FN_PTR(address, BarrierSetRuntime::value_copy), src, dst, inline_layout_info);
}
}
void BarrierSetAssembler::copy_load_at(MacroAssembler* masm,
DecoratorSet decorators,
BasicType type,
@ -389,8 +416,8 @@ void BarrierSetAssembler::check_oop(MacroAssembler* masm, Register obj, Register
__ cbnz(tmp1, error);
// make sure klass is 'reasonable', which is not zero.
__ load_klass(obj, obj); // get klass
__ cbz(obj, error); // if klass is null it is broken
__ load_narrow_klass(tmp1, obj); // get klass
__ cbz(tmp1, error); // if klass is null it is broken
}
void BarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembler* masm, Register weak_handle, Register obj, Register tmp, Label& slow_path) {

3
src/hotspot/cpu/aarch64/gc/shared/barrierSetAssembler_aarch64.hpp
View File

@ -99,6 +99,9 @@ public:
virtual void store_at(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
Address dst, Register val, Register tmp1, Register tmp2, Register tmp3);
virtual void flat_field_copy(MacroAssembler* masm, DecoratorSet decorators,
Register src, Register dst, Register inline_layout_info);
virtual void try_resolve_jobject_in_native(MacroAssembler* masm, Register jni_env,
Register obj, Register tmp, Label& slowpath);

182
src/hotspot/cpu/aarch64/gc/shared/barrierSetNMethod_aarch64.cpp
View File

@ -22,6 +22,7 @@
*
*/
#include "asm/macroAssembler.hpp"
#include "code/codeCache.hpp"
#include "code/nativeInst.hpp"
#include "gc/shared/barrierSet.hpp"
@ -60,83 +61,160 @@ static int entry_barrier_offset(nmethod* nm) {
return 0;
}
static int* decode_guard_from_instruction(nmethod* nm, address& instruction) {
int* result = reinterpret_cast<int*>(MacroAssembler::target_addr_for_insn(instruction));
assert(nm->insts_contains(reinterpret_cast<address>(result)) ||
nm->stub_contains(reinterpret_cast<address>(result)),
"guard must be in nmethod code");
return result;
}
// The NativeNMethodBarrier class encapsulates up to three entrypoints and handles their
// arming/verification.
// An entrypoint is defined as a tuple of <instr. address, guard address>:
// * The instr. address corresponds to the ldr of the guard value of that entrypoint.
// * The guard address is the address where the guard value of that entrypoint resides.
//
// Each nmethod has at least one entrypoint. The default must always be well-defined
// (neither instruction nor guard are nullptr).
//
// When using the scalarized calling convention, up to two additional (verified) entrypoints,
// alt1 and alt2 can be present. The meaning of these depends on who compiled the nmethod.
//
// The mapping of C1-compiled methods (scalarization used) looks as follows:
// * alt1: verified entry point
// * alt2 (optional): verified inline ro entry point
//
// The mapping of C2-compiled methods (scalarization used) looks as follows:
// * alt1: verified inline entry point
// * alt2 (optional): verified inline ro entry point
//
// In other scenarios, neither alt1 nor alt2 are defined.
class NativeNMethodBarrier {
address _instruction_address;
int* _guard_addr;
nmethod* _nm;
private:
// The addresses of the instructions that act as the guards.
address _default_entry_instruction;
address _verified_alt1_instruction;
address _verified_alt2_instruction;
// Pointers representing the actual guard values themselves.
int* _default_entry_guard;
int* _verified_alt1_guard;
int* _verified_alt2_guard;
address instruction_address() const { return _instruction_address; }
public:
NativeNMethodBarrier(nmethod* nm) :
_default_entry_instruction(nullptr),
_verified_alt1_instruction(nullptr),
_verified_alt2_instruction(nullptr),
_default_entry_guard(nullptr),
_verified_alt1_guard(nullptr),
_verified_alt2_guard(nullptr)
{
// The default entry point has a known address. The guard address can be
// decoded from the literal in the instruction. Verification will confirm
// that this instruction corresponds to a load.
_default_entry_instruction = nm->code_begin() + nm->frame_complete_offset() + entry_barrier_offset(nm);
_default_entry_guard = decode_guard_from_instruction(nm, _default_entry_instruction);
int *guard_addr() {
return _guard_addr;
}
// If the nmethod has scalarized arguments, then there are more entry
// points, each with their own nmethod entry barrier.
if (!nm->is_osr_method() && nm->method()->has_scalarized_args()) {
assert(nm->verified_entry_point() != nm->verified_inline_entry_point(), "scalarized entry point not found");
address method_body = nm->is_compiled_by_c1() ? nm->verified_inline_entry_point() : nm->verified_entry_point();
int barrier_offset = _default_entry_instruction - method_body;
int local_guard_offset(nmethod* nm) {
// It's the last instruction
return (-entry_barrier_offset(nm)) - 4;
}
// Set the first alternative entry point.
address entry_point2 = nm->is_compiled_by_c1() ? nm->verified_entry_point() : nm->verified_inline_entry_point();
_verified_alt1_instruction = entry_point2 + barrier_offset;
assert(_default_entry_instruction != _verified_alt1_instruction, "sanity");
_verified_alt1_guard = decode_guard_from_instruction(nm, _verified_alt1_instruction);
public:
NativeNMethodBarrier(nmethod* nm): _nm(nm) {
_instruction_address = nm->code_begin() + nm->frame_complete_offset() + entry_barrier_offset(nm);
if (nm->is_compiled_by_c2()) {
// With c2 compiled code, the guard is out-of-line in a stub
// We find it using the RelocIterator.
RelocIterator iter(nm);
while (iter.next()) {
if (iter.type() == relocInfo::entry_guard_type) {
entry_guard_Relocation* const reloc = iter.entry_guard_reloc();
_guard_addr = reinterpret_cast<int*>(reloc->addr());
return;
}
}
ShouldNotReachHere();
// If there is a second alternative entry point, set it too.
if (method_body != nm->verified_inline_ro_entry_point() && entry_point2 != nm->verified_inline_ro_entry_point()) {
_verified_alt2_instruction = nm->verified_inline_ro_entry_point() + barrier_offset;
_verified_alt2_guard = decode_guard_from_instruction(nm, _verified_alt2_instruction);
assert(_default_entry_instruction != _verified_alt2_instruction &&
_verified_alt1_instruction != _verified_alt2_instruction,
"sanity");
}
_guard_addr = reinterpret_cast<int*>(instruction_address() + local_guard_offset(nm));
}
// Perform the checking as verification.
err_msg msg("%s", "");
assert(check_barriers(msg), "%s", msg.buffer());
}
int get_value() {
return AtomicAccess::load_acquire(guard_addr());
// Gets the value of the default entry guard.
// This does not consider the alternative entrypoints, as these should
// all be consistent. It is up to the caller to enforce this.
int get_default_guard_value() {
return AtomicAccess::load_acquire(_default_entry_guard);
}
void set_value(int value, int bit_mask) {
// Sets the value for all barriers.
void set_values(int value, int bit_mask) {
set_value_impl(_default_entry_guard, value, bit_mask);
if (_verified_alt1_guard != nullptr) {
set_value_impl(_verified_alt1_guard, value, bit_mask);
}
if (_verified_alt2_guard != nullptr) {
set_value_impl(_verified_alt2_guard, value, bit_mask);
}
}
// Verifies that all potential barriers are correct.
bool check_barriers(err_msg& msg) {
// The default entry barrier should always be checked.
if (!check_barrier_impl(_default_entry_instruction, msg)) {
return false;
}
// Check the alternative entry barriers only if they are specified.
// Note that the guard values are already validated at construction time,
// if they fall out of the nmethod range, this will be caught earlier.
if (_verified_alt1_instruction != nullptr &&
!check_barrier_impl(_verified_alt1_instruction, msg)) {
return false;
}
if (_verified_alt2_instruction != nullptr &&
!check_barrier_impl(_verified_alt2_instruction, msg)) {
return false;
}
return true;
}
private:
// Sets the value for a single barrier.
void set_value_impl(int* guard, int value, int bit_mask) {
if (bit_mask == ~0) {
AtomicAccess::release_store(guard_addr(), value);
AtomicAccess::release_store(guard, value);
return;
}
assert((value & ~bit_mask) == 0, "trying to set bits outside the mask");
value &= bit_mask;
int old_value = AtomicAccess::load(guard_addr());
int old_value = AtomicAccess::load(guard);
while (true) {
// Only bits in the mask are changed
int new_value = value | (old_value & ~bit_mask);
if (new_value == old_value) break;
int v = AtomicAccess::cmpxchg(guard_addr(), old_value, new_value, memory_order_release);
int v = AtomicAccess::cmpxchg(guard, old_value, new_value, memory_order_release);
if (v == old_value) break;
old_value = v;
}
}
bool check_barrier(err_msg& msg) const;
void verify() const {
err_msg msg("%s", "");
assert(check_barrier(msg), "%s", msg.buffer());
// Checks the validity of a single barrier.
// The first instruction of the nmethod entry barrier is an ldrw (literal)
// instruction. Verify that it's really there, so the offsets are not skewed.
bool check_barrier_impl(address& instruction, err_msg& msg) {
NativeInstruction* ni = nativeInstruction_at(instruction);
if (!ni->is_ldrw_gpr_literal()) {
msg.print("Nmethod entry barrier did not start with ldrw (literal) as expected. "
"Addr: " PTR_FORMAT " Code: " UINT32_FORMAT, p2i(instruction), ni->encoding());
return false;
}
return true;
}
};
// The first instruction of the nmethod entry barrier is an ldrw (literal)
// instruction. Verify that it's really there, so the offsets are not skewed.
bool NativeNMethodBarrier::check_barrier(err_msg& msg) const {
NativeInstruction* ni = nativeInstruction_at(instruction_address());
if (!ni->is_ldrw_gpr_literal()) {
msg.print("Nmethod entry barrier did not start with ldrw (literal) as expected. "
"Addr: " PTR_FORMAT " Code: " UINT32_FORMAT, p2i(instruction_address()), ni->encoding());
return false;
}
return true;
}
/* We're called from an nmethod when we need to deoptimize it. We do
this by throwing away the nmethod's frame and jumping to the
ic_miss stub. This looks like there has been an IC miss at the
@ -197,7 +275,7 @@ void BarrierSetNMethod::set_guard_value(nmethod* nm, int value, int bit_mask) {
MACOS_AARCH64_ONLY(ThreadWXEnable wx(WXWrite, Thread::current()));
NativeNMethodBarrier barrier(nm);
barrier.set_value(value, bit_mask);
barrier.set_values(value, bit_mask);
}
int BarrierSetNMethod::guard_value(nmethod* nm) {
@ -206,5 +284,5 @@ int BarrierSetNMethod::guard_value(nmethod* nm) {
}
NativeNMethodBarrier barrier(nm);
return barrier.get_value();
return barrier.get_default_guard_value();
}

203
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp
View File

@ -420,6 +420,32 @@ void ShenandoahBarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembl
__ bind(done);
}
void ShenandoahBarrierSetAssembler::check_oop(MacroAssembler* masm, Register obj, Register tmp1, Register tmp2, Label& L_error) {
// Check if the oop is in the right area of memory
__ mov(tmp2, (intptr_t) Universe::verify_oop_mask());
__ andr(tmp1, obj, tmp2);
__ mov(tmp2, (intptr_t) Universe::verify_oop_bits());
// Compare tmp1 and tmp2. We don't use a compare
// instruction here because the flags register is live.
__ eor(tmp1, tmp1, tmp2);
__ cbnz(tmp1, L_error);
// This routine is sometimes called before applying GC barriers.
// With +COH, loading the klass may end up loading forwarding pointer instead.
Label L_skip;
if (UseCompactObjectHeaders) {
Address gc_state(rthread, in_bytes(ShenandoahThreadLocalData::gc_state_offset()));
__ ldrb(tmp1, gc_state);
__ tbnz(tmp1, ShenandoahHeap::HAS_FORWARDED_BITPOS, L_skip);
}
// Make sure klass is 'reasonable', which is not zero.
__ load_narrow_klass(tmp1, obj);
__ cbz(tmp1, L_error);
__ bind(L_skip);
}
void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
Register start, Register count, Register scratch) {
assert(ShenandoahCardBarrier, "Should have been checked by caller");
@ -456,79 +482,38 @@ void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssemb
#define __ ce->masm()->
void ShenandoahBarrierSetAssembler::gen_pre_barrier_stub(LIR_Assembler* ce, ShenandoahPreBarrierStub* stub) {
ShenandoahBarrierSetC1* bs = (ShenandoahBarrierSetC1*)BarrierSet::barrier_set()->barrier_set_c1();
// At this point we know that marking is in progress.
// If do_load() is true then we have to emit the
// load of the previous value; otherwise it has already
// been loaded into _pre_val.
void ShenandoahBarrierSetAssembler::keepalive_barrier_c1_stub(LIR_Assembler* ce, ShenandoahKeepaliveBarrierStub* stub) {
__ bind(*stub->entry());
assert(stub->pre_val()->is_register(), "Precondition.");
ShenandoahBarrierSetC1* bs = (ShenandoahBarrierSetC1*)BarrierSet::barrier_set()->barrier_set_c1();
Register pre_val_reg = stub->pre_val()->as_register();
Register obj = stub->obj()->as_register();
if (stub->do_load()) {
ce->mem2reg(stub->addr(), stub->pre_val(), T_OBJECT, stub->patch_code(), stub->info(), false /*wide*/);
ce->mem2reg(stub->addr(), stub->obj(), T_OBJECT, lir_patch_none, nullptr, /* wide = */ false);
}
__ cbz(pre_val_reg, *stub->continuation());
ce->store_parameter(stub->pre_val()->as_register(), 0);
__ far_call(RuntimeAddress(bs->pre_barrier_c1_runtime_code_blob()->code_begin()));
__ cbz(obj, *stub->continuation());
ce->store_parameter(obj, 0);
__ far_call(RuntimeAddress(bs->keepalive_barrier_stub()));
__ b(*stub->continuation());
}
void ShenandoahBarrierSetAssembler::gen_load_reference_barrier_stub(LIR_Assembler* ce, ShenandoahLoadReferenceBarrierStub* stub) {
ShenandoahBarrierSetC1* bs = (ShenandoahBarrierSetC1*)BarrierSet::barrier_set()->barrier_set_c1();
void ShenandoahBarrierSetAssembler::load_reference_barrier_c1_stub(LIR_Assembler* ce, ShenandoahLoadReferenceBarrierStub* stub) {
__ bind(*stub->entry());
DecoratorSet decorators = stub->decorators();
bool is_strong = ShenandoahBarrierSet::is_strong_access(decorators);
bool is_weak = ShenandoahBarrierSet::is_weak_access(decorators);
bool is_phantom = ShenandoahBarrierSet::is_phantom_access(decorators);
bool is_native = ShenandoahBarrierSet::is_native_access(decorators);
ShenandoahBarrierSetC1* bs = (ShenandoahBarrierSetC1*)BarrierSet::barrier_set()->barrier_set_c1();
Register obj = stub->obj()->as_register();
Register res = stub->result()->as_register();
Register addr = stub->addr()->as_pointer_register();
Register tmp1 = stub->tmp1()->as_register();
Register tmp2 = stub->tmp2()->as_register();
Register slow_result = stub->slow_result()->as_register();
assert_different_registers(obj, addr, slow_result);
assert(slow_result == r0, "C1 must know about our slow call result register");
assert(res == r0, "result must arrive in r0");
if (res != obj) {
__ mov(res, obj);
}
if (is_strong) {
// Check for object in cset.
if (AOTCodeCache::is_on_for_dump()) {
__ lea(tmp2, ExternalAddress(AOTRuntimeConstants::cset_base_address()));
__ ldr(tmp2, Address(tmp2));
__ lea(tmp1, ExternalAddress(AOTRuntimeConstants::grain_shift_address()));
__ ldrw(tmp1, Address(tmp1));
__ lsrv(tmp1, res, tmp1);
} else {
__ mov(tmp2, ShenandoahHeap::in_cset_fast_test_addr());
__ lsr(tmp1, res, ShenandoahHeapRegion::region_size_bytes_shift_jint());
}
__ ldrb(tmp2, Address(tmp2, tmp1));
__ cbz(tmp2, *stub->continuation());
}
ce->store_parameter(res, 0);
ce->store_parameter(obj, 0);
ce->store_parameter(addr, 1);
if (is_strong) {
if (is_native) {
__ far_call(RuntimeAddress(bs->load_reference_barrier_strong_native_rt_code_blob()->code_begin()));
} else {
__ far_call(RuntimeAddress(bs->load_reference_barrier_strong_rt_code_blob()->code_begin()));
}
} else if (is_weak) {
__ far_call(RuntimeAddress(bs->load_reference_barrier_weak_rt_code_blob()->code_begin()));
} else {
assert(is_phantom, "only remaining strength");
__ far_call(RuntimeAddress(bs->load_reference_barrier_phantom_rt_code_blob()->code_begin()));
__ far_call(RuntimeAddress(bs->load_reference_barrier_stub(stub->decorators())));
if (obj != slow_result) {
__ mov(obj, slow_result);
}
__ b(*stub->continuation());
@ -538,89 +523,27 @@ void ShenandoahBarrierSetAssembler::gen_load_reference_barrier_stub(LIR_Assemble
#define __ sasm->
void ShenandoahBarrierSetAssembler::generate_c1_pre_barrier_runtime_stub(StubAssembler* sasm) {
__ prologue("shenandoah_pre_barrier", false);
// arg0 : previous value of memory
BarrierSet* bs = BarrierSet::barrier_set();
const Register pre_val = r0;
const Register thread = rthread;
const Register tmp = rscratch1;
Address queue_index(thread, in_bytes(ShenandoahThreadLocalData::satb_mark_queue_index_offset()));
Address buffer(thread, in_bytes(ShenandoahThreadLocalData::satb_mark_queue_buffer_offset()));
Label done;
Label runtime;
// Is marking still active?
Address gc_state(thread, in_bytes(ShenandoahThreadLocalData::gc_state_offset()));
__ ldrb(tmp, gc_state);
__ tbz(tmp, ShenandoahHeap::MARKING_BITPOS, done);
// Can we store original value in the thread's buffer?
__ ldr(tmp, queue_index);
__ cbz(tmp, runtime);
__ sub(tmp, tmp, wordSize);
__ str(tmp, queue_index);
__ ldr(rscratch2, buffer);
__ add(tmp, tmp, rscratch2);
__ load_parameter(0, rscratch2);
__ str(rscratch2, Address(tmp, 0));
__ b(done);
__ bind(runtime);
__ push_call_clobbered_registers();
__ load_parameter(0, pre_val);
__ call_VM_leaf(CAST_FROM_FN_PTR(address, ShenandoahRuntime::write_barrier_pre), pre_val);
__ pop_call_clobbered_registers();
__ bind(done);
void ShenandoahBarrierSetAssembler::keepalive_barrier_c1_runtime_stub(StubAssembler* sasm) {
__ prologue("shenandoah_keepalive_barrier", false);
const Register tmp_obj = r0;
const Register tmp1 = r1;
const Register tmp2 = r2;
__ push(RegSet::of(tmp1, tmp2, tmp_obj), sp);
__ load_parameter(0, tmp_obj);
satb_barrier(sasm, noreg, tmp_obj, rthread, tmp1, tmp2);
__ pop(RegSet::of(tmp1, tmp2, tmp_obj), sp);
__ epilogue();
}
void ShenandoahBarrierSetAssembler::generate_c1_load_reference_barrier_runtime_stub(StubAssembler* sasm, DecoratorSet decorators) {
void ShenandoahBarrierSetAssembler::load_reference_barrier_c1_runtime_stub(StubAssembler* sasm, DecoratorSet decorators) {
__ prologue("shenandoah_load_reference_barrier", false);
// arg0 : object to be resolved
__ push_call_clobbered_registers();
__ load_parameter(0, r0);
__ load_parameter(1, r1);
bool is_strong = ShenandoahBarrierSet::is_strong_access(decorators);
bool is_weak = ShenandoahBarrierSet::is_weak_access(decorators);
bool is_phantom = ShenandoahBarrierSet::is_phantom_access(decorators);
bool is_native = ShenandoahBarrierSet::is_native_access(decorators);
if (is_strong) {
if (is_native) {
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong)));
} else {
if (UseCompressedOops) {
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong_narrow)));
} else {
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_strong)));
}
}
} else if (is_weak) {
assert(!is_native, "weak must not be called off-heap");
if (UseCompressedOops) {
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_weak_narrow)));
} else {
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_weak)));
}
} else {
assert(is_phantom, "only remaining strength");
assert(is_native, "phantom must only be called off-heap");
__ lea(lr, RuntimeAddress(CAST_FROM_FN_PTR(address, ShenandoahRuntime::load_reference_barrier_phantom)));
}
__ blr(lr);
__ mov(rscratch1, r0);
__ pop_call_clobbered_registers();
__ mov(r0, rscratch1);
const Register tmp_obj = r0;
const Register tmp_addr = r1;
__ push(RegSet::of(tmp_addr), sp);
__ load_parameter(0, tmp_obj);
__ load_parameter(1, tmp_addr);
load_reference_barrier(sasm, tmp_obj, Address(tmp_addr, 0), decorators);
__ pop(RegSet::of(tmp_addr), sp);
__ epilogue();
}
@ -700,8 +623,14 @@ void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, Mac
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, tmp1, tmp2, tmp3, narrow);
atomic_memory_order order = acquire ? memory_order_seq_cst : memory_order_release;
// CAS!
__ cmpxchg(addr, oldval, newval, op_size, acquire, /* release */ true, weak, exchange ? res : noreg);
if (weak) {
__ cmpxchg_weak(addr, oldval, newval, op_size, order, exchange ? res : noreg);
} else {
__ cmpxchg(addr, oldval, newval, op_size, order, exchange ? res : noreg);
}
// If we need a boolean result out of CAS, set the flag appropriately and promote the result.
if (!exchange) {

12
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp
View File

@ -32,7 +32,7 @@
#include "gc/shenandoah/shenandoahBarrierSet.hpp"
#ifdef COMPILER1
class LIR_Assembler;
class ShenandoahPreBarrierStub;
class ShenandoahKeepaliveBarrierStub;
class ShenandoahLoadReferenceBarrierStub;
class StubAssembler;
#endif
@ -74,12 +74,14 @@ public:
Register obj, Register tmp, Label& slowpath);
virtual void try_peek_weak_handle_in_nmethod(MacroAssembler* masm, Register weak_handle, Register obj,
Register tmp, Label& slow_path);
virtual void check_oop(MacroAssembler* masm, Register obj, Register tmp1, Register tmp2, Label& L_error);
#ifdef COMPILER1
void gen_pre_barrier_stub(LIR_Assembler* ce, ShenandoahPreBarrierStub* stub);
void gen_load_reference_barrier_stub(LIR_Assembler* ce, ShenandoahLoadReferenceBarrierStub* stub);
void generate_c1_pre_barrier_runtime_stub(StubAssembler* sasm);
void generate_c1_load_reference_barrier_runtime_stub(StubAssembler* sasm, DecoratorSet decorators);
void keepalive_barrier_c1_stub(LIR_Assembler* ce, ShenandoahKeepaliveBarrierStub* stub);
void keepalive_barrier_c1_runtime_stub(StubAssembler* sasm);
void load_reference_barrier_c1_stub(LIR_Assembler* ce, ShenandoahLoadReferenceBarrierStub* stub);
void load_reference_barrier_c1_runtime_stub(StubAssembler* sasm, DecoratorSet decorators);
#endif
#ifdef COMPILER2

10
src/hotspot/cpu/aarch64/gc/z/zBarrierSetAssembler_aarch64.cpp
View File

@ -283,10 +283,7 @@ void ZBarrierSetAssembler::store_barrier_medium(MacroAssembler* masm,
// If we get this far, we know there is a young raw null value in the field.
__ relocate(barrier_Relocation::spec(), ZBarrierRelocationFormatStoreGoodBeforeMov);
__ movzw(rtmp1, barrier_Relocation::unpatched);
__ cmpxchg(rtmp2, zr, rtmp1,
Assembler::xword,
false /* acquire */, false /* release */, true /* weak */,
rtmp3);
__ cmpxchg_weak(rtmp2, zr, rtmp1, Assembler::xword, memory_order_relaxed, rtmp3);
__ br(Assembler::NE, slow_path);
__ bind(slow_path_continuation);
@ -1388,9 +1385,8 @@ void ZBarrierSetAssembler::check_oop(MacroAssembler* masm, Register obj, Registe
__ bind(check_oop);
// make sure klass is 'reasonable', which is not zero.
__ load_klass(tmp1, obj); // get klass
__ tst(tmp1, tmp1);
__ br(Assembler::EQ, error); // if klass is null it is broken
__ load_narrow_klass(tmp1, obj); // get narrow klass
__ cbz(tmp1, error); // if klass is null it is broken
__ bind(check_zaddress);
// Check if the oop is in the right area of memory

10
src/hotspot/cpu/aarch64/gc/z/z_aarch64.ad
View File

@ -207,8 +207,7 @@ instruct zCompareAndSwapP(iRegINoSp res, indirect mem, iRegP oldval, iRegP newva
Address ref_addr($mem$$Register);
z_store_barrier(masm, this, ref_addr, $newval$$Register, $newval_tmp$$Register, rscratch2, true /* is_atomic */);
z_color(masm, this, $oldval_tmp$$Register, $oldval$$Register);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword,
false /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword, memory_order_release);
__ cset($res$$Register, Assembler::EQ);
%}
@ -231,8 +230,7 @@ instruct zCompareAndSwapPAcq(iRegINoSp res, indirect mem, iRegP oldval, iRegP ne
Address ref_addr($mem$$Register);
z_store_barrier(masm, this, ref_addr, $newval$$Register, $newval_tmp$$Register, rscratch2, true /* is_atomic */);
z_color(masm, this, $oldval_tmp$$Register, $oldval$$Register);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword,
true /* acquire */, true /* release */, false /* weak */, noreg);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword, memory_order_seq_cst);
__ cset($res$$Register, Assembler::EQ);
%}
@ -255,7 +253,7 @@ instruct zCompareAndExchangeP(iRegPNoSp res, indirect mem, iRegP oldval, iRegP n
z_store_barrier(masm, this, ref_addr, $newval$$Register, $newval_tmp$$Register, rscratch2, true /* is_atomic */);
z_color(masm, this, $oldval_tmp$$Register, $oldval$$Register);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword,
false /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_release, $res$$Register);
z_uncolor(masm, this, $res$$Register);
%}
@ -278,7 +276,7 @@ instruct zCompareAndExchangePAcq(iRegPNoSp res, indirect mem, iRegP oldval, iReg
z_store_barrier(masm, this, ref_addr, $newval$$Register, $newval_tmp$$Register, rscratch2, true /* is_atomic */);
z_color(masm, this, $oldval_tmp$$Register, $oldval$$Register);
__ cmpxchg($mem$$Register, $oldval_tmp$$Register, $newval_tmp$$Register, Assembler::xword,
true /* acquire */, true /* release */, false /* weak */, $res$$Register);
memory_order_seq_cst, $res$$Register);
z_uncolor(masm, this, $res$$Register);
%}

2
src/hotspot/cpu/aarch64/globalDefinitions_aarch64.hpp
View File

@ -61,7 +61,9 @@ const bool CCallingConventionRequiresIntsAsLongs = false;
// evidence that it's worth doing.
#define DEOPTIMIZE_WHEN_PATCHING
#if !defined(_WINDOWS)
#define SUPPORT_RESERVED_STACK_AREA
#endif
#if defined(__APPLE__) || defined(_WIN64)
#define R18_RESERVED

5
src/hotspot/cpu/aarch64/globals_aarch64.hpp
View File

@ -48,7 +48,7 @@ define_pd_global(intx, OptoLoopAlignment, 16);
// stack if compiled for unix and LP64. To pass stack overflow tests we need
// 20 shadow pages.
#define DEFAULT_STACK_SHADOW_PAGES (20 DEBUG_ONLY(+5))
#define DEFAULT_STACK_RESERVED_PAGES (1)
#define DEFAULT_STACK_RESERVED_PAGES (NOT_WINDOWS(1) WINDOWS_ONLY(0))
#define MIN_STACK_YELLOW_PAGES DEFAULT_STACK_YELLOW_PAGES
#define MIN_STACK_RED_PAGES DEFAULT_STACK_RED_PAGES
@ -67,6 +67,9 @@ define_pd_global(bool, RewriteFrequentPairs, true);
define_pd_global(bool, PreserveFramePointer, false);
define_pd_global(bool, InlineTypePassFieldsAsArgs, true);
define_pd_global(bool, InlineTypeReturnedAsFields, true);
define_pd_global(uintx, TypeProfileLevel, 111);
define_pd_global(bool, CompactStrings, true);

198
src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp
View File

@ -32,9 +32,11 @@
#include "interpreter/interpreterRuntime.hpp"
#include "logging/log.hpp"
#include "oops/arrayOop.hpp"
#include "oops/constMethodFlags.hpp"
#include "oops/markWord.hpp"
#include "oops/method.hpp"
#include "oops/methodData.hpp"
#include "oops/inlineKlass.hpp"
#include "oops/resolvedFieldEntry.hpp"
#include "oops/resolvedIndyEntry.hpp"
#include "oops/resolvedMethodEntry.hpp"
@ -208,6 +210,39 @@ void InterpreterMacroAssembler::get_method_counters(Register method,
bind(has_counters);
}
void InterpreterMacroAssembler::read_flat_field(Register entry, Register obj) {
call_VM(obj, CAST_FROM_FN_PTR(address, InterpreterRuntime::read_flat_field), obj, entry);
}
void InterpreterMacroAssembler::write_flat_field(Register entry, Register field_offset,
Register tmp1, Register tmp2,
Register obj) {
assert_different_registers(entry, field_offset, tmp1, tmp2, obj);
Label slow_path, done;
load_unsigned_byte(tmp1, Address(entry, in_bytes(ResolvedFieldEntry::flags_offset())));
test_field_is_not_null_free_inline_type(tmp1, noreg /* temp */, slow_path);
null_check(r0); // FIXME JDK-8341120
add(obj, obj, field_offset);
load_klass(tmp1, r0, tmp2);
payload_address(r0, r0, tmp1);
Register layout_info = field_offset;
load_unsigned_short(tmp1, Address(entry, in_bytes(ResolvedFieldEntry::field_index_offset())));
ldr(tmp2, Address(entry, in_bytes(ResolvedFieldEntry::field_holder_offset())));
inline_layout_info(tmp2, tmp1, layout_info);
flat_field_copy(IN_HEAP, r0, obj, layout_info);
b(done);
bind(slow_path);
call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::write_flat_field), obj, r0, entry);
bind(done);
}
// Load object from cpool->resolved_references(index)
void InterpreterMacroAssembler::load_resolved_reference_at_index(
Register result, Register index, Register tmp) {
@ -242,13 +277,15 @@ void InterpreterMacroAssembler::load_resolved_klass_at_offset(
// Kills:
// r2
void InterpreterMacroAssembler::gen_subtype_check(Register Rsub_klass,
Label& ok_is_subtype) {
Label& ok_is_subtype,
bool profile) {
assert(Rsub_klass != r0, "r0 holds superklass");
assert(Rsub_klass != r2, "r2 holds 2ndary super array length");
// Profile the not-null value's klass.
profile_typecheck(r2, Rsub_klass); // blows r2
if (profile) {
profile_typecheck(r2, Rsub_klass); // blows r2
}
// Do the check.
check_klass_subtype(Rsub_klass, r0, r2, ok_is_subtype); // blows r2
}
@ -658,6 +695,37 @@ void InterpreterMacroAssembler::remove_activation(TosState state,
bind(no_reserved_zone_enabling);
}
if (state == atos && InlineTypeReturnedAsFields) {
Label skip;
Label not_null;
cbnz(r0, not_null);
// Returned value is null, zero all return registers because they may belong to oop fields
mov(j_rarg1, zr);
mov(j_rarg2, zr);
mov(j_rarg3, zr);
mov(j_rarg4, zr);
mov(j_rarg5, zr);
mov(j_rarg6, zr);
mov(j_rarg7, zr);
b(skip);
bind(not_null);
// Check if we are returning a non-null inline type and load its fields into registers
test_oop_is_not_inline_type(r0, rscratch2, skip, /* can_be_null= */ false);
// Load fields from a buffered value with an inline class specific handler
load_klass(rscratch1 /*dst*/, r0 /*src*/, rscratch2 /*tmp*/);
ldr(rscratch1, Address(rscratch1, InlineKlass::adr_members_offset()));
ldr(rscratch1, Address(rscratch1, InlineKlass::unpack_handler_offset()));
// Unpack handler can be null if inline type is not scalarizable in returns
cbz(rscratch1, skip);
blr(rscratch1);
bind(skip);
// Check above kills sender esp in rscratch2. Reload it.
ldr(rscratch2, Address(rfp, frame::interpreter_frame_sender_sp_offset * wordSize));
}
// remove frame anchor
leave();
@ -935,7 +1003,7 @@ void InterpreterMacroAssembler::profile_taken_branch(Register mdp) {
}
void InterpreterMacroAssembler::profile_not_taken_branch(Register mdp) {
void InterpreterMacroAssembler::profile_not_taken_branch(Register mdp, bool acmp) {
if (ProfileInterpreter) {
Label profile_continue;
@ -947,7 +1015,7 @@ void InterpreterMacroAssembler::profile_not_taken_branch(Register mdp) {
// The method data pointer needs to be updated to correspond to
// the next bytecode
update_mdp_by_constant(mdp, in_bytes(BranchData::branch_data_size()));
update_mdp_by_constant(mdp, acmp ? in_bytes(ACmpData::acmp_data_size()) : in_bytes(BranchData::branch_data_size()));
bind(profile_continue);
}
}
@ -1133,6 +1201,120 @@ void InterpreterMacroAssembler::profile_switch_case(Register index,
}
}
template <class ArrayData> void InterpreterMacroAssembler::profile_array_type(Register mdp,
Register array,
Register tmp) {
if (ProfileInterpreter) {
Label profile_continue;
// If no method data exists, go to profile_continue.
test_method_data_pointer(mdp, profile_continue);
mov(tmp, array);
profile_obj_type(tmp, Address(mdp, in_bytes(ArrayData::array_offset())));
Label not_flat;
test_non_flat_array_oop(array, tmp, not_flat);
set_mdp_flag_at(mdp, ArrayData::flat_array_byte_constant());
bind(not_flat);
Label not_null_free;
test_non_null_free_array_oop(array, tmp, not_null_free);
set_mdp_flag_at(mdp, ArrayData::null_free_array_byte_constant());
bind(not_null_free);
bind(profile_continue);
}
}
template void InterpreterMacroAssembler::profile_array_type<ArrayLoadData>(Register mdp,
Register array,
Register tmp);
template void InterpreterMacroAssembler::profile_array_type<ArrayStoreData>(Register mdp,
Register array,
Register tmp);
void InterpreterMacroAssembler::profile_multiple_element_types(Register mdp, Register element, Register tmp, const Register tmp2) {
if (ProfileInterpreter) {
Label profile_continue;
// If no method data exists, go to profile_continue.
test_method_data_pointer(mdp, profile_continue);
Label done, update;
cbnz(element, update);
set_mdp_flag_at(mdp, BitData::null_seen_byte_constant());
b(done);
bind(update);
load_klass(tmp, element, tmp2);
// Record the object type.
profile_receiver_type(tmp, mdp, 0);
bind(done);
// The method data pointer needs to be updated.
update_mdp_by_constant(mdp, in_bytes(ArrayStoreData::array_store_data_size()));
bind(profile_continue);
}
}
void InterpreterMacroAssembler::profile_element_type(Register mdp,
Register element,
Register tmp) {
if (ProfileInterpreter) {
Label profile_continue;
// If no method data exists, go to profile_continue.
test_method_data_pointer(mdp, profile_continue);
mov(tmp, element);
profile_obj_type(tmp, Address(mdp, in_bytes(ArrayLoadData::element_offset())));
// The method data pointer needs to be updated.
update_mdp_by_constant(mdp, in_bytes(ArrayLoadData::array_load_data_size()));
bind(profile_continue);
}
}
void InterpreterMacroAssembler::profile_acmp(Register mdp,
Register left,
Register right,
Register tmp) {
if (ProfileInterpreter) {
Label profile_continue;
// If no method data exists, go to profile_continue.
test_method_data_pointer(mdp, profile_continue);
mov(tmp, left);
profile_obj_type(tmp, Address(mdp, in_bytes(ACmpData::left_offset())));
Label left_not_inline_type;
test_oop_is_not_inline_type(left, tmp, left_not_inline_type);
set_mdp_flag_at(mdp, ACmpData::left_inline_type_byte_constant());
bind(left_not_inline_type);
mov(tmp, right);
profile_obj_type(tmp, Address(mdp, in_bytes(ACmpData::right_offset())));
Label right_not_inline_type;
test_oop_is_not_inline_type(right, tmp, right_not_inline_type);
set_mdp_flag_at(mdp, ACmpData::right_inline_type_byte_constant());
bind(right_not_inline_type);
bind(profile_continue);
}
}
void InterpreterMacroAssembler::_interp_verify_oop(Register reg, TosState state, const char* file, int line) {
if (state == atos) {
MacroAssembler::_verify_oop_checked(reg, "broken oop", file, line);
@ -1403,7 +1585,7 @@ void InterpreterMacroAssembler::profile_obj_type(Register obj, const Address& md
b(next);
bind(update);
load_klass(obj, obj);
load_klass(obj, obj, rscratch1);
ldr(rscratch1, mdo_addr);
eor(obj, obj, rscratch1);
@ -1503,7 +1685,7 @@ void InterpreterMacroAssembler::profile_arguments_type(Register mdp, Register ca
// argument. tmp is the number of cells left in the
// CallTypeData/VirtualCallTypeData to reach its end. Non null
// if there's a return to profile.
assert(ReturnTypeEntry::static_cell_count() < TypeStackSlotEntries::per_arg_count(), "can't move past ret type");
assert(SingleTypeEntry::static_cell_count() < TypeStackSlotEntries::per_arg_count(), "can't move past ret type");
add(mdp, mdp, tmp, LSL, exact_log2(DataLayout::cell_size));
}
str(mdp, Address(rfp, frame::interpreter_frame_mdp_offset * wordSize));
@ -1549,7 +1731,7 @@ void InterpreterMacroAssembler::profile_return_type(Register mdp, Register ret,
bind(do_profile);
}
Address mdo_ret_addr(mdp, -in_bytes(ReturnTypeEntry::size()));
Address mdo_ret_addr(mdp, -in_bytes(SingleTypeEntry::size()));
mov(tmp, ret);
profile_obj_type(tmp, mdo_ret_addr);

18
src/hotspot/cpu/aarch64/interp_masm_aarch64.hpp
View File

@ -158,6 +158,16 @@ class InterpreterMacroAssembler: public MacroAssembler {
void get_cache_index_at_bcp(Register index, int bcp_offset, size_t index_size = sizeof(u2));
void get_method_counters(Register method, Register mcs, Label& skip);
// Allocate instance in "obj" and read in the content of the inline field
// NOTES:
// - input holder object via "obj", which must be r0,
// will return new instance via the same reg
void read_flat_field(Register entry, Register obj);
void write_flat_field(Register entry, Register field_offset,
Register tmp1, Register tmp2,
Register obj);
// load cpool->resolved_references(index);
void load_resolved_reference_at_index(Register result, Register index, Register tmp = r5);
@ -202,7 +212,7 @@ class InterpreterMacroAssembler: public MacroAssembler {
// Generate a subtype check: branch to ok_is_subtype if sub_klass is
// a subtype of super_klass.
void gen_subtype_check( Register sub_klass, Label &ok_is_subtype );
void gen_subtype_check( Register sub_klass, Label &ok_is_subtype, bool profile = true);
// Dispatching
void dispatch_prolog(TosState state, int step = 0);
@ -282,7 +292,7 @@ class InterpreterMacroAssembler: public MacroAssembler {
void narrow(Register result);
void profile_taken_branch(Register mdp);
void profile_not_taken_branch(Register mdp);
void profile_not_taken_branch(Register mdp, bool acmp = false);
void profile_call(Register mdp);
void profile_final_call(Register mdp);
void profile_virtual_call(Register receiver, Register mdp);
@ -293,6 +303,10 @@ class InterpreterMacroAssembler: public MacroAssembler {
void profile_switch_default(Register mdp);
void profile_switch_case(Register index_in_scratch, Register mdp,
Register scratch2);
template <class ArrayData> void profile_array_type(Register mdp, Register array, Register tmp);
void profile_multiple_element_types(Register mdp, Register element, Register tmp, Register tmp2);
void profile_element_type(Register mdp, Register element, Register tmp);
void profile_acmp(Register mdp, Register left, Register right, Register tmp);
void profile_obj_type(Register obj, const Address& mdo_addr);
void profile_arguments_type(Register mdp, Register callee, Register tmp, bool is_virtual);

5
src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
View File

@ -257,6 +257,11 @@ class SlowSignatureHandler
}
}
virtual void pass_valuetype() {
// values are handled with oops, like objects
pass_object();
}
virtual void pass_long() {
intptr_t value = *double_slot_addr();
if (pass_gpr(value) < 0) {

3
src/hotspot/cpu/aarch64/jniFastGetField_aarch64.cpp
View File

@ -30,6 +30,7 @@
#include "prims/jniFastGetField.hpp"
#include "prims/jvm_misc.hpp"
#include "prims/jvmtiExport.hpp"
#include "runtime/jfieldIDWorkaround.hpp"
#include "runtime/javaThread.inline.hpp"
#include "runtime/safepoint.hpp"
#include "runtime/threadWXSetters.inline.hpp"
@ -152,7 +153,7 @@ address JNI_FastGetField::generate_fast_get_int_field0(BasicType type) {
BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler();
bs->try_resolve_jobject_in_native(masm, c_rarg0, robj, rscratch1, slow);
__ lsr(roffset, c_rarg2, 2); // offset
__ lsr(roffset, c_rarg2, jfieldIDWorkaround::offset_shift); // offset
__ add(result, robj, roffset);
assert(count < LIST_CAPACITY, "LIST_CAPACITY too small");

987
src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
View File

File diff suppressed because it is too large Load Diff

147
src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
View File

@ -32,11 +32,18 @@
#include "metaprogramming/enableIf.hpp"
#include "oops/compressedOops.hpp"
#include "oops/compressedKlass.hpp"
#include "runtime/atomicAccess.hpp"
#include "runtime/vm_version.hpp"
#include "utilities/globalDefinitions.hpp"
#include "utilities/macros.hpp"
#include "utilities/powerOfTwo.hpp"
#include "runtime/signature.hpp"
class ciInlineKlass;
class OopMap;
struct GtestFriendToMacroAssembler;
// MacroAssembler extends Assembler by frequently used macros.
//
@ -45,6 +52,7 @@ class OopMap;
class MacroAssembler: public Assembler {
friend class LIR_Assembler;
friend struct GtestFriendToMacroAssembler;
public:
using Assembler::mov;
@ -90,28 +98,31 @@ class MacroAssembler: public Assembler {
void call_VM_helper(Register oop_result, address entry_point, int number_of_arguments, bool check_exceptions = true);
private:
enum KlassDecodeMode {
KlassDecodeNone,
KlassDecodeZero,
KlassDecodeXor,
KlassDecodeMovk
KlassDecodeMovk,
KlassDecodeFallback
};
// Calculate decoding mode based on given parameters, used for checking then ultimately setting.
static KlassDecodeMode klass_decode_mode(address base, int shift, const size_t range);
private:
static KlassDecodeMode _klass_decode_mode;
// Returns above setting with asserts
static KlassDecodeMode klass_decode_mode();
public:
// Checks the decode mode and returns false if not compatible with preferred decoding mode.
static bool check_klass_decode_mode(address base, int shift, const size_t range);
// Calculate decoding mode based on given parameters, used for checking then ultimately setting.
static KlassDecodeMode klass_decode_mode(address base, int shift, const size_t range);
// Sets the decode mode and returns false if cannot be set.
static bool set_klass_decode_mode(address base, int shift, const size_t range);
void emit_encode_klass_not_null(Register dst, Register src, Register tmp,
address base, int shift, KlassDecodeMode decode_mode);
void emit_decode_klass_not_null(Register dst, Register src, Register tmp,
address base, int shift, KlassDecodeMode decode_mode);
public:
// Determines the decode mode best suited for the given encoding parameters.
static void initialize_klass_decode_mode(address base, int shift, const size_t range);
public:
MacroAssembler(CodeBuffer* code) : Assembler(code) {}
@ -185,7 +196,8 @@ class MacroAssembler: public Assembler {
void strw(Register Rx, const Address &adr);
// Frame creation and destruction shared between JITs.
void build_frame(int framesize);
DEBUG_ONLY(void build_frame(int framesize);)
void build_frame(int framesize DEBUG_ONLY(COMMA bool zap_rfp_lr_spills));
void remove_frame(int framesize);
virtual void _call_Unimplemented(address call_site) {
@ -307,19 +319,27 @@ class MacroAssembler: public Assembler {
}
inline void lslw(Register Rd, Register Rn, unsigned imm) {
ubfmw(Rd, Rn, ((32 - imm) & 31), (31 - imm));
if (imm > 0 || Rd != Rn) {
ubfmw(Rd, Rn, ((32 - imm) & 31), (31 - imm));
}
}
inline void lsl(Register Rd, Register Rn, unsigned imm) {
ubfm(Rd, Rn, ((64 - imm) & 63), (63 - imm));
if (imm > 0 || Rd != Rn) {
ubfm(Rd, Rn, ((64 - imm) & 63), (63 - imm));
}
}
inline void lsrw(Register Rd, Register Rn, unsigned imm) {
ubfmw(Rd, Rn, imm, 31);
if (imm > 0 || Rd != Rn) {
ubfmw(Rd, Rn, imm, 31);
}
}
inline void lsr(Register Rd, Register Rn, unsigned imm) {
ubfm(Rd, Rn, imm, 63);
if (imm > 0 || Rd != Rn) {
ubfm(Rd, Rn, imm, 63);
}
}
inline void rorw(Register Rd, Register Rn, unsigned imm) {
@ -696,6 +716,26 @@ public:
static bool needs_explicit_null_check(intptr_t offset);
static bool uses_implicit_null_check(void* address);
// markWord tests, kills markWord reg
void test_markword_is_inline_type(Register markword, Label& is_inline_type);
// inlineKlass queries, kills temp_reg
void test_oop_is_not_inline_type(Register object, Register tmp, Label& not_inline_type, bool can_be_null = true);
void test_field_is_null_free_inline_type(Register flags, Register temp_reg, Label& is_null_free);
void test_field_is_not_null_free_inline_type(Register flags, Register temp_reg, Label& not_null_free);
void test_field_is_flat(Register flags, Register temp_reg, Label& is_flat);
// Check oops for special arrays, i.e. flat arrays and/or null-free arrays
void test_oop_prototype_bit(Register oop, Register temp_reg, int32_t test_bit, bool jmp_set, Label& jmp_label);
void test_flat_array_oop(Register klass, Register temp_reg, Label& is_flat_array);
void test_non_flat_array_oop(Register oop, Register temp_reg, Label&is_non_flat_array);
void test_null_free_array_oop(Register oop, Register temp_reg, Label& is_null_free_array);
void test_non_null_free_array_oop(Register oop, Register temp_reg, Label&is_non_null_free_array);
// Check array klass layout helper for flat or null-free arrays...
void test_flat_array_layout(Register lh, Label& is_flat_array);
static address target_addr_for_insn(address insn_addr);
// Required platform-specific helpers for Label::patch_instructions.
@ -922,10 +962,13 @@ public:
void load_method_holder(Register holder, Register method);
// oop manipulations
void load_metadata(Register dst, Register src);
void load_narrow_klass_compact(Register dst, Register src);
void load_klass(Register dst, Register src);
void store_klass(Register dst, Register src);
void cmp_klass(Register obj, Register klass, Register tmp);
void load_narrow_klass(Register dst, Register src);
void load_klass(Register dst, Register src, Register tmp);
void store_klass(Register dst, Register src, Register tmp);
void cmp_klass(Register obj, Register klass, Register tmp, Register tmp2);
void cmp_klasses_from_objects(Register obj1, Register obj2, Register tmp1, Register tmp2);
void resolve_weak_handle(Register result, Register tmp1, Register tmp2);
@ -938,6 +981,12 @@ public:
void access_store_at(BasicType type, DecoratorSet decorators, Address dst, Register val,
Register tmp1, Register tmp2, Register tmp3);
void flat_field_copy(DecoratorSet decorators, Register src, Register dst, Register inline_layout_info);
// inline type data payload offsets...
void payload_offset(Register inline_klass, Register offset);
void payload_address(Register oop, Register data, Register inline_klass);
void load_heap_oop(Register dst, Address src, Register tmp1,
Register tmp2, DecoratorSet decorators = 0);
@ -951,6 +1000,8 @@ public:
// stored using routines that take a jobject.
void store_heap_oop_null(Address dst);
void load_prototype_header(Register dst, Register src);
void store_klass_gap(Register dst, Register src);
// This dummy is to prevent a call to store_heap_oop from
@ -970,12 +1021,8 @@ public:
void set_narrow_oop(Register dst, jobject obj);
void decode_klass_not_null_for_aot(Register dst, Register src);
void encode_klass_not_null_for_aot(Register dst, Register src);
void encode_klass_not_null(Register r);
void decode_klass_not_null(Register r);
void encode_klass_not_null(Register dst, Register src);
void decode_klass_not_null(Register dst, Register src);
void encode_klass_not_null(Register dst, Register src, Register tmp);
void decode_klass_not_null(Register dst, Register src, Register tmp);
void set_narrow_klass(Register dst, Klass* k);
@ -1000,6 +1047,7 @@ public:
void java_round_float(Register dst, FloatRegister src, FloatRegister ftmp);
// allocation
void tlab_allocate(
Register obj, // result: pointer to object after successful allocation
Register var_size_in_bytes, // object size in bytes if unknown at compile time; invalid otherwise
@ -1010,6 +1058,8 @@ public:
);
void verify_tlab();
void inline_layout_info(Register holder_klass, Register index, Register layout_info);
// interface method calling
void lookup_interface_method(Register recv_klass,
Register intf_klass,
@ -1239,12 +1289,25 @@ public:
str(rscratch1, adr);
}
private:
// A generic CAS; success or failure is in the EQ flag.
// Clobbers rscratch1
void cmpxchg(Register addr, Register expected, Register new_val,
enum operand_size size,
bool acquire, bool release, bool weak,
Register result);
enum operand_size size, enum atomic_memory_order order,
bool weak, Register result);
public:
void cmpxchg(Register addr, Register expected, Register new_val,
enum operand_size size, enum atomic_memory_order order,
Register result = noreg) {
cmpxchg(addr, expected, new_val, size, order, /* weak */ false, result);
}
void cmpxchg_weak(Register addr, Register expected, Register new_val,
enum operand_size size, enum atomic_memory_order order,
Register result = noreg) {
cmpxchg(addr, expected, new_val, size, order, /* weak */ true, result);
}
#ifdef ASSERT
// Template short-hand support to clean-up after a failed call to trampoline
@ -1339,15 +1402,16 @@ public:
static bool far_branches() {
return ReservedCodeCacheSize > branch_range;
}
// Check if branches to the non nmethod section require a far jump
// Check if the static call stub branch needs a far jump.
static bool codestub_branch_needs_far_jump() {
if (AOTCodeCache::is_on_for_dump()) {
// To calculate far_codestub_branch_size correctly.
// To calculate static_call_stub_size correctly.
return true;
}
return CodeCache::max_distance_to_non_nmethod() > branch_range;
return far_branches();
}
// Check if a branch to the given address needs a far jump.
static bool target_needs_far_branch(address addr);
// Emit a direct call/jump if the entry address will always be in range,
// otherwise a far call/jump.
@ -1359,18 +1423,10 @@ public:
// In the case of a far call/jump, the entry address is put in the tmp register.
// The tmp register is invalidated.
//
// Far_jump returns the amount of the emitted code.
void far_call(Address entry, Register tmp = rscratch1);
// Far_jump returns the amount of the emitted code.
int far_jump(Address entry, Register tmp = rscratch1);
static int far_codestub_branch_size() {
if (codestub_branch_needs_far_jump()) {
return 3 * 4; // adrp, add, br
} else {
return 4;
}
}
// Emit the CompiledIC call idiom
address ic_call(address entry, jint method_index = 0);
static int ic_check_size();
@ -1467,6 +1523,13 @@ public:
void adrp(Register reg1, const Address &dest, uint64_t &byte_offset);
void verified_entry(Compile* C, int sp_inc);
// Inline type specific methods
#include "asm/macroAssembler_common.hpp"
void save_stack_increment(int sp_inc, int frame_size);
void tableswitch(Register index, jint lowbound, jint highbound,
Label &jumptable, Label &jumptable_end, int stride = 1) {
adr(rscratch1, jumptable);
@ -1541,6 +1604,8 @@ public:
void string_equals(Register a1, Register a2, Register result, Register cnt1);
void fill_words(Register base, Register cnt, Register value);
void fill_words(Register base, uint64_t cnt, Register value);
address zero_words(Register base, uint64_t cnt);
address zero_words(Register ptr, Register cnt);
void zero_dcache_blocks(Register base, Register cnt);
@ -1790,6 +1855,8 @@ public:
SVE_DESTRUCTIVE_BINARY_5(sve_fmul, sve_fsub, sve_lsl, sve_lsr, sve_mul)
SVE_DESTRUCTIVE_BINARY_5(sve_orr, sve_smax, sve_smin, sve_sqadd, sve_sqsub)
SVE_DESTRUCTIVE_BINARY_5(sve_sub, sve_uqadd, sve_uqsub, sve_umax, sve_umin)
SVE_DESTRUCTIVE_BINARY_INS(sve_sdiv);
SVE_DESTRUCTIVE_BINARY_INS(sve_udiv);
#undef SVE_DESTRUCTIVE_BINARY_INS
#undef SVE_DESTRUCTIVE_BINARY_5

12
src/hotspot/cpu/aarch64/methodHandles_aarch64.cpp
View File

@ -76,7 +76,7 @@ void MethodHandles::verify_klass(MacroAssembler* _masm,
__ verify_oop(obj);
__ cbz(obj, L_bad);
__ push(RegSet::of(temp, temp2), sp);
__ load_klass(temp, obj);
__ load_klass(temp, obj, temp2);
__ cmpptr(temp, ExternalAddress((address) klass_addr));
__ br(Assembler::EQ, L_ok);
intptr_t super_check_offset = klass->super_check_offset();
@ -161,7 +161,11 @@ void MethodHandles::jump_from_method_handle(MacroAssembler* _masm, Register meth
__ BIND(run_compiled_code);
}
const ByteSize entry_offset = for_compiler_entry ? Method::from_compiled_offset() :
// The following jump might pass an inline type argument that was erased to Object as oop to a
// callee that expects inline type arguments to be passed as fields. We need to call the compiled
// value entry (_code->inline_entry_point() or _adapter->c2i_inline_entry()) which will take care
// of translating between the calling conventions.
const ByteSize entry_offset = for_compiler_entry ? Method::from_compiled_inline_offset() :
Method::from_interpreted_offset();
__ ldr(rscratch1,Address(method, entry_offset));
__ br(rscratch1);
@ -368,7 +372,7 @@ void MethodHandles::generate_method_handle_dispatch(MacroAssembler* _masm,
__ null_check(receiver_reg);
} else {
// load receiver klass itself
__ load_klass(temp1_recv_klass, receiver_reg);
__ load_klass(temp1_recv_klass, receiver_reg, temp2);
__ verify_klass_ptr(temp1_recv_klass);
}
BLOCK_COMMENT("check_receiver {");
@ -376,7 +380,7 @@ void MethodHandles::generate_method_handle_dispatch(MacroAssembler* _masm,
// Check the receiver against the MemberName.clazz
if (VerifyMethodHandles && iid == vmIntrinsics::_linkToSpecial) {
// Did not load it above...
__ load_klass(temp1_recv_klass, receiver_reg);
__ load_klass(temp1_recv_klass, receiver_reg, temp2);
__ verify_klass_ptr(temp1_recv_klass);
}
if (VerifyMethodHandles && iid != vmIntrinsics::_linkToInterface) {

9
src/hotspot/cpu/aarch64/register_aarch64.hpp
View File

@ -28,6 +28,7 @@
#include "asm/register.hpp"
#include "utilities/checkedCast.hpp"
#include "utilities/globalDefinitions.hpp"
#include "utilities/powerOfTwo.hpp"
class VMRegImpl;
@ -513,25 +514,25 @@ template<int N> bool vs_write_before_read(const VSeq<N>& vout, const VSeq<N>& vi
template<int N>
VSeq<N/2> vs_front(const VSeq<N>& v) {
static_assert(N > 0 && ((N & 1) == 0), "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
return VSeq<N/2>(v.base(), v.delta());
}
template<int N>
VSeq<N/2> vs_back(const VSeq<N>& v) {
static_assert(N > 0 && ((N & 1) == 0), "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
return VSeq<N/2>(v.base() + N / 2 * v.delta(), v.delta());
}
template<int N>
VSeq<N/2> vs_even(const VSeq<N>& v) {
static_assert(N > 0 && ((N & 1) == 0), "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
return VSeq<N/2>(v.base(), v.delta() * 2);
}
template<int N>
VSeq<N/2> vs_odd(const VSeq<N>& v) {
static_assert(N > 0 && ((N & 1) == 0), "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
return VSeq<N/2>(v.base() + v.delta(), v.delta() * 2);
}

976
src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
View File

File diff suppressed because it is too large Load Diff

34
src/hotspot/cpu/aarch64/stackChunkFrameStream_aarch64.inline.hpp
View File

@ -35,8 +35,30 @@ template <ChunkFrames frame_kind>
inline bool StackChunkFrameStream<frame_kind>::is_in_frame(void* p0) const {
assert(!is_done(), "");
intptr_t* p = (intptr_t*)p0;
int argsize = is_compiled() ? (_cb->as_nmethod()->num_stack_arg_slots() * VMRegImpl::stack_slot_size) >> LogBytesPerWord : 0;
int frame_size = _cb->frame_size() + argsize;
int frame_size = _cb->frame_size();
if (is_compiled()) {
nmethod* nm = _cb->as_nmethod_or_null();
if (nm->needs_stack_repair() && nm->is_compiled_by_c2()) {
frame f = to_frame();
bool augmented = f.was_augmented_on_entry(frame_size);
if (!augmented) {
// Fix: C2 caller, so frame was not extended and thus the
// size read from the frame does not include the arguments.
// Ideally we have to count the arg size for the scalarized
// convention. For now we include the size of the caller frame
// which would at least be equal to that.
RegisterMap map(nullptr,
RegisterMap::UpdateMap::skip,
RegisterMap::ProcessFrames::skip,
RegisterMap::WalkContinuation::skip);
frame caller = to_frame().sender(&map);
assert(caller.is_compiled_frame() && caller.cb()->as_nmethod()->is_compiled_by_c2(), "needs stack repair but was not extended with c1/interpreter caller");
frame_size += (caller.real_fp() - caller.sp());
}
} else {
frame_size += _cb->as_nmethod()->num_stack_arg_slots() * VMRegImpl::stack_slot_size >> LogBytesPerWord;
}
}
return p == sp() - frame::sender_sp_offset || ((p - unextended_sp()) >= 0 && (p - unextended_sp()) < frame_size);
}
#endif
@ -46,7 +68,13 @@ inline frame StackChunkFrameStream<frame_kind>::to_frame() const {
if (is_done()) {
return frame(_sp, _sp, nullptr, nullptr, nullptr, nullptr, true);
} else {
return frame(sp(), unextended_sp(), fp(), pc(), cb(), _oopmap, true);
frame f = frame(sp(), unextended_sp(), fp(), pc(), cb(), _oopmap, true);
// If caller tries to get the sender of this frame and PreserveFramePointer
// is set, fp() will be used which contains the old value at the time of
// freeze (fp is reconstructed again during thaw). Setting sp as trusted
// causes the sender code to use _unextended_sp instead (see sender_for_compiled_frame()).
f.set_sp_is_trusted();
return f;
}
}

109
src/hotspot/cpu/aarch64/stubGenerator_aarch64.cpp
View File

@ -504,20 +504,25 @@ class StubGenerator: public StubCodeGenerator {
// T_OBJECT, T_LONG, T_FLOAT or T_DOUBLE is treated as T_INT)
// n.b. this assumes Java returns an integral result in r0
// and a floating result in j_farg0
__ ldr(j_rarg2, result);
Label is_long, is_float, is_double, exit;
__ ldr(j_rarg1, result_type);
__ cmp(j_rarg1, (u1)T_OBJECT);
// All of j_rargN may be used to return inline type fields so be careful
// not to clobber those.
// SharedRuntime::generate_buffered_inline_type_adapter() knows the register
// assignment of Rresult below.
Register Rresult = r14, Rresult_type = r15;
__ ldr(Rresult, result);
Label is_long, is_float, is_double, check_prim, exit;
__ ldr(Rresult_type, result_type);
__ cmp(Rresult_type, (u1)T_OBJECT);
__ br(Assembler::EQ, check_prim);
__ cmp(Rresult_type, (u1)T_LONG);
__ br(Assembler::EQ, is_long);
__ cmp(j_rarg1, (u1)T_LONG);
__ br(Assembler::EQ, is_long);
__ cmp(j_rarg1, (u1)T_FLOAT);
__ cmp(Rresult_type, (u1)T_FLOAT);
__ br(Assembler::EQ, is_float);
__ cmp(j_rarg1, (u1)T_DOUBLE);
__ cmp(Rresult_type, (u1)T_DOUBLE);
__ br(Assembler::EQ, is_double);
// handle T_INT case
__ strw(r0, Address(j_rarg2));
__ strw(r0, Address(Rresult));
__ BIND(exit);
@ -569,17 +574,28 @@ class StubGenerator: public StubCodeGenerator {
__ ret(lr);
// handle return types different from T_INT
__ BIND(check_prim);
if (InlineTypeReturnedAsFields) {
// Check for scalarized return value
__ tbz(r0, 0, is_long);
// Load pack handler address
__ andr(rscratch1, r0, -2);
__ ldr(rscratch1, Address(rscratch1, InlineKlass::adr_members_offset()));
__ ldr(rscratch1, Address(rscratch1, InlineKlass::pack_handler_jobject_offset()));
__ blr(rscratch1);
__ b(exit);
}
__ BIND(is_long);
__ str(r0, Address(j_rarg2, 0));
__ str(r0, Address(Rresult, 0));
__ br(Assembler::AL, exit);
__ BIND(is_float);
__ strs(j_farg0, Address(j_rarg2, 0));
__ strs(j_farg0, Address(Rresult, 0));
__ br(Assembler::AL, exit);
__ BIND(is_double);
__ strd(j_farg0, Address(j_rarg2, 0));
__ strd(j_farg0, Address(Rresult, 0));
__ br(Assembler::AL, exit);
// record the stub entry and end plus the auxiliary entry
@ -2258,7 +2274,7 @@ class StubGenerator: public StubCodeGenerator {
// checked.
assert_different_registers(from, to, count, ckoff, ckval, start_to,
copied_oop, r19_klass, count_save);
copied_oop, r19_klass, count_save, rscratch1);
__ align(CodeEntryAlignment);
StubCodeMark mark(this, stub_id);
@ -2342,7 +2358,7 @@ class StubGenerator: public StubCodeGenerator {
gct1);
__ cbz(copied_oop, L_store_element);
__ load_klass(r19_klass, copied_oop);// query the object klass
__ load_klass(r19_klass, copied_oop, rscratch1);// query the object klass
BLOCK_COMMENT("type_check:");
generate_type_check(/*sub_klass*/r19_klass,
@ -2568,7 +2584,7 @@ class StubGenerator: public StubCodeGenerator {
__ movw(scratch_length, length); // length (elements count, 32-bits value)
__ tbnz(scratch_length, 31, L_failed); // i.e. sign bit set
__ load_klass(scratch_src_klass, src);
__ load_narrow_klass(scratch_src_klass, src);
#ifdef ASSERT
// assert(src->klass() != nullptr);
{
@ -2578,11 +2594,12 @@ class StubGenerator: public StubCodeGenerator {
__ bind(L1);
__ stop("broken null klass");
__ bind(L2);
__ load_klass(rscratch1, dst);
__ load_narrow_klass(rscratch1, dst);
__ cbz(rscratch1, L1); // this would be broken also
BLOCK_COMMENT("} assert klasses not null done");
}
#endif
__ decode_klass_not_null(scratch_src_klass, scratch_src_klass, rscratch1);
// Load layout helper (32-bits)
//
@ -2602,10 +2619,16 @@ class StubGenerator: public StubCodeGenerator {
__ cbzw(rscratch2, L_objArray);
// if (src->klass() != dst->klass()) return -1;
__ load_klass(rscratch2, dst);
__ load_klass(rscratch2, dst, rscratch1);
__ eor(rscratch2, rscratch2, scratch_src_klass);
__ cbnz(rscratch2, L_failed);
// Check for flat inline type array -> return -1
__ test_flat_array_oop(src, rscratch2, L_failed);
// Check for null-free (non-flat) inline type array -> handle as object array
__ test_null_free_array_oop(src, rscratch2, L_objArray);
// if (!src->is_Array()) return -1;
__ tbz(lh, 31, L_failed); // i.e. (lh >= 0)
@ -2698,7 +2721,7 @@ class StubGenerator: public StubCodeGenerator {
Label L_plain_copy, L_checkcast_copy;
// test array classes for subtyping
__ load_klass(r15, dst);
__ load_klass(r15, dst, rscratch1);
__ cmp(scratch_src_klass, r15); // usual case is exact equality
__ br(Assembler::NE, L_checkcast_copy);
@ -2727,7 +2750,7 @@ class StubGenerator: public StubCodeGenerator {
arraycopy_range_checks(src, src_pos, dst, dst_pos, scratch_length,
r15, L_failed);
__ load_klass(dst_klass, dst); // reload
__ load_klass(dst_klass, dst, rscratch1); // reload
// Marshal the base address arguments now, freeing registers.
__ lea(from, Address(src, src_pos, Address::lsl(LogBytesPerHeapOop)));
@ -5442,6 +5465,7 @@ class StubGenerator: public StubCodeGenerator {
// address supplied in base.
template<int N>
void vs_ldpq(const VSeq<N>& v, Register base) {
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N; i += 2) {
__ ldpq(v[i], v[i+1], Address(base, 16 * i));
}
@ -5452,7 +5476,7 @@ class StubGenerator: public StubCodeGenerator {
// in base using post-increment addressing
template<int N>
void vs_ldpq_post(const VSeq<N>& v, Register base) {
static_assert((N & (N - 1)) == 0, "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N; i += 2) {
__ ldpq(v[i], v[i+1], __ post(base, 32));
}
@ -5463,7 +5487,7 @@ class StubGenerator: public StubCodeGenerator {
// supplied in base using post-increment addressing
template<int N>
void vs_stpq_post(const VSeq<N>& v, Register base) {
static_assert((N & (N - 1)) == 0, "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N; i += 2) {
__ stpq(v[i], v[i+1], __ post(base, 32));
}
@ -5474,7 +5498,7 @@ class StubGenerator: public StubCodeGenerator {
// using post-increment addressing.
template<int N>
void vs_ld2_post(const VSeq<N>& v, Assembler::SIMD_Arrangement T, Register base) {
static_assert((N & (N - 1)) == 0, "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N; i += 2) {
__ ld2(v[i], v[i+1], T, __ post(base, 32));
}
@ -5485,7 +5509,7 @@ class StubGenerator: public StubCodeGenerator {
// post-increment addressing.
template<int N>
void vs_st2_post(const VSeq<N>& v, Assembler::SIMD_Arrangement T, Register base) {
static_assert((N & (N - 1)) == 0, "sequence length must be even");
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N; i += 2) {
__ st2(v[i], v[i+1], T, __ post(base, 32));
}
@ -5530,6 +5554,7 @@ class StubGenerator: public StubCodeGenerator {
// offsets array
template<int N>
void vs_ldpq_indexed(const VSeq<N>& v, Register base, int start, int (&offsets)[N/2]) {
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N/2; i++) {
__ ldpq(v[2*i], v[2*i+1], Address(base, start + offsets[i]));
}
@ -5577,6 +5602,7 @@ class StubGenerator: public StubCodeGenerator {
template<int N>
void vs_ld2_indexed(const VSeq<N>& v, Assembler::SIMD_Arrangement T, Register base,
Register tmp, int start, int (&offsets)[N/2]) {
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N/2; i++) {
__ add(tmp, base, start + offsets[i]);
__ ld2(v[2*i], v[2*i+1], T, tmp);
@ -5590,6 +5616,7 @@ class StubGenerator: public StubCodeGenerator {
template<int N>
void vs_st2_indexed(const VSeq<N>& v, Assembler::SIMD_Arrangement T, Register base,
Register tmp, int start, int (&offsets)[N/2]) {
static_assert(N > 0 && is_even(N), "sequence length must be even");
for (int i = 0; i < N/2; i++) {
__ add(tmp, base, start + offsets[i]);
__ st2(v[2*i], v[2*i+1], T, tmp);
@ -12393,6 +12420,30 @@ class StubGenerator: public StubCodeGenerator {
}
#endif // LINUX
static void save_return_registers(MacroAssembler* masm) {
if (InlineTypeReturnedAsFields) {
masm->push(RegSet::range(r0, r7), sp);
masm->sub(sp, sp, 4 * wordSize);
masm->st1(v0, v1, v2, v3, masm->T1D, Address(sp));
masm->sub(sp, sp, 4 * wordSize);
masm->st1(v4, v5, v6, v7, masm->T1D, Address(sp));
} else {
masm->fmovd(rscratch1, v0);
masm->stp(rscratch1, r0, Address(masm->pre(sp, -2 * wordSize)));
}
}
static void restore_return_registers(MacroAssembler* masm) {
if (InlineTypeReturnedAsFields) {
masm->ld1(v4, v5, v6, v7, masm->T1D, Address(masm->post(sp, 4 * wordSize)));
masm->ld1(v0, v1, v2, v3, masm->T1D, Address(masm->post(sp, 4 * wordSize)));
masm->pop(RegSet::range(r0, r7), sp);
} else {
masm->ldp(rscratch1, r0, Address(masm->post(sp, 2 * wordSize)));
masm->fmovd(v0, rscratch1);
}
}
address generate_cont_thaw(Continuation::thaw_kind kind) {
bool return_barrier = Continuation::is_thaw_return_barrier(kind);
bool return_barrier_exception = Continuation::is_thaw_return_barrier_exception(kind);
@ -12407,8 +12458,7 @@ class StubGenerator: public StubCodeGenerator {
if (return_barrier) {
// preserve possible return value from a method returning to the return barrier
__ fmovd(rscratch1, v0);
__ stp(rscratch1, r0, Address(__ pre(sp, -2 * wordSize)));
save_return_registers(_masm);
}
__ movw(c_rarg1, (return_barrier ? 1 : 0));
@ -12417,8 +12467,7 @@ class StubGenerator: public StubCodeGenerator {
if (return_barrier) {
// restore return value (no safepoint in the call to thaw, so even an oop return value should be OK)
__ ldp(rscratch1, r0, Address(__ post(sp, 2 * wordSize)));
__ fmovd(v0, rscratch1);
restore_return_registers(_masm);
}
assert_asm(_masm, (__ ldr(rscratch1, Address(rthread, JavaThread::cont_entry_offset())), __ cmp(sp, rscratch1)), Assembler::EQ, "incorrect sp");
@ -12437,8 +12486,7 @@ class StubGenerator: public StubCodeGenerator {
if (return_barrier) {
// save original return value -- again
__ fmovd(rscratch1, v0);
__ stp(rscratch1, r0, Address(__ pre(sp, -2 * wordSize)));
save_return_registers(_masm);
}
// If we want, we can templatize thaw by kind, and have three different entries
@ -12449,8 +12497,7 @@ class StubGenerator: public StubCodeGenerator {
if (return_barrier) {
// restore return value (no safepoint in the call to thaw, so even an oop return value should be OK)
__ ldp(rscratch1, r0, Address(__ post(sp, 2 * wordSize)));
__ fmovd(v0, rscratch1);
restore_return_registers(_masm);
} else {
__ mov(r0, zr); // return 0 (success) from doYield
}

14
src/hotspot/cpu/aarch64/templateInterpreterGenerator_aarch64.cpp
View File

@ -41,6 +41,7 @@
#include "oops/methodCounters.hpp"
#include "oops/methodData.hpp"
#include "oops/oop.inline.hpp"
#include "oops/inlineKlass.hpp"
#include "oops/resolvedIndyEntry.hpp"
#include "oops/resolvedMethodEntry.hpp"
#include "prims/jvmtiExport.hpp"
@ -467,6 +468,11 @@ address TemplateInterpreterGenerator::generate_return_entry_for(TosState state,
__ lea(esp, Address(rfp, rscratch1, Address::lsl(Interpreter::logStackElementSize)));
// and null it as marker that esp is now tos until next java call
__ str(zr, Address(rfp, frame::interpreter_frame_last_sp_offset * wordSize));
if (state == atos && InlineTypeReturnedAsFields) {
__ store_inline_type_fields_to_buf(nullptr, true);
}
__ restore_bcp();
__ restore_locals();
__ restore_constant_pool_cache();
@ -1633,7 +1639,7 @@ address TemplateInterpreterGenerator::generate_native_entry(bool synchronized) {
//
// Generic interpreted method entry to (asm) interpreter
//
address TemplateInterpreterGenerator::generate_normal_entry(bool synchronized) {
address TemplateInterpreterGenerator::generate_normal_entry(bool synchronized, bool object_init) {
// determine code generation flags
bool inc_counter = UseCompiler || CountCompiledCalls;
@ -1760,6 +1766,12 @@ address TemplateInterpreterGenerator::generate_normal_entry(bool synchronized) {
#endif
}
// Issue a StoreStore barrier on entry to Object_init if the
// class has strict field fields. Be lazy, always do it.
if (object_init) {
__ membar(MacroAssembler::StoreStore);
}
// start execution
#ifdef ASSERT
{

340
src/hotspot/cpu/aarch64/templateTable_aarch64.cpp
View File

@ -43,6 +43,7 @@
#include "oops/resolvedMethodEntry.hpp"
#include "prims/jvmtiExport.hpp"
#include "prims/methodHandles.hpp"
#include "runtime/arguments.hpp"
#include "runtime/frame.inline.hpp"
#include "runtime/sharedRuntime.hpp"
#include "runtime/stubRoutines.hpp"
@ -169,6 +170,7 @@ void TemplateTable::patch_bytecode(Bytecodes::Code bc, Register bc_reg,
Label L_patch_done;
switch (bc) {
case Bytecodes::_fast_vputfield:
case Bytecodes::_fast_aputfield:
case Bytecodes::_fast_bputfield:
case Bytecodes::_fast_zputfield:
@ -753,7 +755,7 @@ void TemplateTable::index_check(Register array, Register index)
}
Label ok;
__ br(Assembler::LO, ok);
// ??? convention: move array into r3 for exception message
// ??? convention: move array into r3 for exception message
__ mov(r3, array);
__ mov(rscratch1, Interpreter::_throw_ArrayIndexOutOfBoundsException_entry);
__ br(rscratch1);
@ -816,11 +818,23 @@ void TemplateTable::aaload()
// r0: array
// r1: index
index_check(r0, r1); // leaves index in r1, kills rscratch1
__ add(r1, r1, arrayOopDesc::base_offset_in_bytes(T_OBJECT) >> LogBytesPerHeapOop);
do_oop_load(_masm,
Address(r0, r1, Address::uxtw(LogBytesPerHeapOop)),
r0,
IS_ARRAY);
__ profile_array_type<ArrayLoadData>(r2, r0, r4);
if (UseArrayFlattening) {
Label is_flat_array, done;
__ test_flat_array_oop(r0, rscratch1 /*temp*/, is_flat_array);
__ add(r1, r1, arrayOopDesc::base_offset_in_bytes(T_OBJECT) >> LogBytesPerHeapOop);
do_oop_load(_masm, Address(r0, r1, Address::uxtw(LogBytesPerHeapOop)), r0, IS_ARRAY);
__ b(done);
__ bind(is_flat_array);
__ call_VM(r0, CAST_FROM_FN_PTR(address, InterpreterRuntime::flat_array_load), r0, r1);
__ bind(done);
} else {
__ add(r1, r1, arrayOopDesc::base_offset_in_bytes(T_OBJECT) >> LogBytesPerHeapOop);
do_oop_load(_masm, Address(r0, r1, Address::uxtw(LogBytesPerHeapOop)), r0, IS_ARRAY);
}
__ profile_element_type(r2, r0, r4);
}
void TemplateTable::baload()
@ -1107,32 +1121,45 @@ void TemplateTable::dastore() {
}
void TemplateTable::aastore() {
Label is_null, ok_is_subtype, done;
Label is_null, is_flat_array, ok_is_subtype, done;
transition(vtos, vtos);
// stack: ..., array, index, value
__ ldr(r0, at_tos()); // value
__ ldr(r2, at_tos_p1()); // index
__ ldr(r3, at_tos_p2()); // array
Address element_address(r3, r4, Address::uxtw(LogBytesPerHeapOop));
index_check(r3, r2); // kills r1
__ profile_array_type<ArrayStoreData>(r4, r3, r5);
__ profile_multiple_element_types(r4, r0, r5, r6);
__ add(r4, r2, arrayOopDesc::base_offset_in_bytes(T_OBJECT) >> LogBytesPerHeapOop);
Address element_address(r3, r4, Address::uxtw(LogBytesPerHeapOop));
// Be careful not to clobber r4 below
// do array store check - check for null value first
__ cbz(r0, is_null);
// Move array class to r5
__ load_klass(r5, r3, rscratch1);
if (UseArrayFlattening) {
__ ldrw(r6, Address(r5, Klass::layout_helper_offset()));
__ test_flat_array_layout(r6, is_flat_array);
}
// Move subklass into r1
__ load_klass(r1, r0);
// Move superklass into r0
__ load_klass(r0, r3);
__ ldr(r0, Address(r0,
ObjArrayKlass::element_klass_offset()));
__ load_klass(r1, r0, rscratch1);
// Move array element superklass into r0
__ ldr(r0, Address(r5, ObjArrayKlass::element_klass_offset()));
// Compress array + index*oopSize + 12 into a single register. Frees r2.
// Generate subtype check. Blows r2, r5
// Superklass in r0. Subklass in r1.
__ gen_subtype_check(r1, ok_is_subtype);
// is "r1 <: r0" ? (value subclass <: array element superclass)
__ gen_subtype_check(r1, ok_is_subtype, false);
// Come here on failure
// object is at TOS
@ -1150,11 +1177,37 @@ void TemplateTable::aastore() {
// Have a null in r0, r3=array, r2=index. Store null at ary[idx]
__ bind(is_null);
__ profile_null_seen(r2);
if (Arguments::is_valhalla_enabled()) {
Label is_null_into_value_array_npe, store_null;
if (UseArrayFlattening) {
__ test_flat_array_oop(r3, rscratch1, is_flat_array);
}
// No way to store null in a null-free array
__ test_null_free_array_oop(r3, rscratch1, is_null_into_value_array_npe);
__ b(store_null);
__ bind(is_null_into_value_array_npe);
__ b(ExternalAddress(Interpreter::_throw_NullPointerException_entry));
__ bind(store_null);
}
// Store a null
// Clobbers: r10, r11, r3
do_oop_store(_masm, element_address, noreg, IS_ARRAY);
__ b(done);
if (UseArrayFlattening) {
Label is_type_ok;
__ bind(is_flat_array); // Store non-null value to flat
__ ldr(r0, at_tos()); // value
__ ldr(r3, at_tos_p1()); // index
__ ldr(r2, at_tos_p2()); // array
__ call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::flat_array_store), r0, r2, r3);
}
// Pop stack arguments
__ bind(done);
@ -1173,7 +1226,7 @@ void TemplateTable::bastore()
// Need to check whether array is boolean or byte
// since both types share the bastore bytecode.
__ load_klass(r2, r3);
__ load_klass(r2, r3, rscratch1);
__ ldrw(r2, Address(r2, Klass::layout_helper_offset()));
int diffbit_index = exact_log2(Klass::layout_helper_boolean_diffbit());
Label L_skip;
@ -1961,19 +2014,68 @@ void TemplateTable::if_nullcmp(Condition cc)
__ profile_not_taken_branch(r0);
}
void TemplateTable::if_acmp(Condition cc)
{
void TemplateTable::if_acmp(Condition cc) {
transition(atos, vtos);
// assume branch is more often taken than not (loops use backward branches)
Label not_taken;
Label taken, not_taken;
__ pop_ptr(r1);
__ profile_acmp(r2, r1, r0, r4);
Register is_inline_type_mask = rscratch1;
__ mov(is_inline_type_mask, markWord::inline_type_pattern);
if (Arguments::is_valhalla_enabled()) {
// The substitutability test is only necessary if r1 and r0 are not the same...
__ cmp(r1, r0);
__ br(Assembler::EQ, (cc == equal) ? taken : not_taken);
// ... neither are null...
__ cbz(r1, (cc == equal) ? not_taken : taken);
__ cbz(r0, (cc == equal) ? not_taken : taken);
// ...and both are values...
__ ldr(r2, Address(r1, oopDesc::mark_offset_in_bytes()));
__ andr(r2, r2, is_inline_type_mask);
__ ldr(r4, Address(r0, oopDesc::mark_offset_in_bytes()));
__ andr(r4, r4, is_inline_type_mask);
__ andr(r2, r2, r4);
__ cmp(r2, is_inline_type_mask);
__ br(Assembler::NE, (cc == equal) ? not_taken : taken);
// ...with the same value klass
__ load_metadata(r2, r1);
__ load_metadata(r4, r0);
__ cmp(r2, r4);
__ br(Assembler::NE, (cc == equal) ? not_taken : taken);
// Know both are the same type, let's test for substitutability...
if (cc == equal) {
invoke_is_substitutable(r0, r1, taken, not_taken);
} else {
invoke_is_substitutable(r0, r1, not_taken, taken);
}
__ stop("Not reachable");
}
__ cmpoop(r1, r0);
__ br(j_not(cc), not_taken);
__ bind(taken);
branch(false, false);
__ bind(not_taken);
__ profile_not_taken_branch(r0);
__ profile_not_taken_branch(r0, true);
}
void TemplateTable::invoke_is_substitutable(Register aobj, Register bobj,
Label& is_subst, Label& not_subst) {
__ call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::is_substitutable), aobj, bobj);
// Restored... r0 answer, jmp to outcome...
__ cbz(r0, not_subst);
__ b(is_subst);
}
void TemplateTable::ret() {
transition(vtos, vtos);
locals_index(r1);
@ -2194,7 +2296,7 @@ void TemplateTable::_return(TosState state)
assert(state == vtos, "only valid state");
__ ldr(c_rarg1, aaddress(0));
__ load_klass(r3, c_rarg1);
__ load_klass(r3, c_rarg1, rscratch1);
__ ldrb(r3, Address(r3, Klass::misc_flags_offset()));
Label skip_register_finalizer;
__ tbz(r3, exact_log2(KlassFlags::_misc_has_finalizer), skip_register_finalizer);
@ -2207,7 +2309,8 @@ void TemplateTable::_return(TosState state)
// Issue a StoreStore barrier after all stores but before return
// from any constructor for any class with a final field. We don't
// know if this is a finalizer, so we always do so.
if (_desc->bytecode() == Bytecodes::_return)
if (_desc->bytecode() == Bytecodes::_return
|| _desc->bytecode() == Bytecodes::_return_register_finalizer)
__ membar(MacroAssembler::StoreStore);
if (_desc->bytecode() != Bytecodes::_return_register_finalizer) {
@ -2599,7 +2702,7 @@ void TemplateTable::pop_and_check_object(Register r)
void TemplateTable::getfield_or_static(int byte_no, bool is_static, RewriteControl rc)
{
const Register cache = r4;
const Register cache = r2;
const Register obj = r4;
const Register index = r3;
const Register tos_state = r3;
@ -2609,6 +2712,7 @@ void TemplateTable::getfield_or_static(int byte_no, bool is_static, RewriteContr
resolve_cache_and_index_for_field(byte_no, cache, index);
jvmti_post_field_access(cache, index, is_static, false);
load_resolved_field_entry(obj, cache, tos_state, off, flags, is_static);
if (!is_static) {
@ -2667,12 +2771,39 @@ void TemplateTable::getfield_or_static(int byte_no, bool is_static, RewriteContr
__ cmp(tos_state, (u1)atos);
__ br(Assembler::NE, notObj);
// atos
do_oop_load(_masm, field, r0, IN_HEAP);
__ push(atos);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_agetfield, bc, r1);
if (!Arguments::is_valhalla_enabled()) {
do_oop_load(_masm, field, r0, IN_HEAP);
__ push(atos);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_agetfield, bc, r1);
}
__ b(Done);
} else { // Valhalla
if (is_static) {
__ load_heap_oop(r0, field, rscratch1, rscratch2);
__ push(atos);
__ b(Done);
} else {
Label is_flat;
__ test_field_is_flat(flags, noreg /* temp */, is_flat);
__ load_heap_oop(r0, field, rscratch1, rscratch2);
__ push(atos);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_agetfield, bc, r1);
}
__ b(Done);
__ bind(is_flat);
// field is flat (null-free or nullable with a null-marker)
__ mov(r0, obj);
__ read_flat_field(cache, r0);
__ verify_oop(r0);
__ push(atos);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_vgetfield, bc, r1);
}
__ b(Done);
}
}
__ b(Done);
__ bind(notObj);
__ cmp(tos_state, (u1)itos);
@ -2833,7 +2964,7 @@ void TemplateTable::putfield_or_static(int byte_no, bool is_static, RewriteContr
const Register tos_state = r3;
const Register obj = r2;
const Register off = r19;
const Register flags = r0;
const Register flags = r6;
const Register bc = r4;
resolve_cache_and_index_for_field(byte_no, cache, index);
@ -2841,11 +2972,9 @@ void TemplateTable::putfield_or_static(int byte_no, bool is_static, RewriteContr
load_resolved_field_entry(obj, cache, tos_state, off, flags, is_static);
Label Done;
__ mov(r5, flags);
{
Label notVolatile;
__ tbz(r5, ResolvedFieldEntry::is_volatile_shift, notVolatile);
__ tbz(flags, ResolvedFieldEntry::is_volatile_shift, notVolatile);
__ membar(MacroAssembler::StoreStore | MacroAssembler::LoadStore);
__ bind(notVolatile);
}
@ -2894,15 +3023,55 @@ void TemplateTable::putfield_or_static(int byte_no, bool is_static, RewriteContr
// atos
{
__ pop(atos);
if (!is_static) pop_and_check_object(obj);
// Store into the field
// Clobbers: r10, r11, r3
do_oop_store(_masm, field, r0, IN_HEAP);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_aputfield, bc, r1, true, byte_no);
}
__ b(Done);
if (!Arguments::is_valhalla_enabled()) {
__ pop(atos);
if (!is_static) pop_and_check_object(obj);
// Store into the field
// Clobbers: r10, r11, r3
do_oop_store(_masm, field, r0, IN_HEAP);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_aputfield, bc, r1, true, byte_no);
}
__ b(Done);
} else { // Valhalla
__ pop(atos);
if (is_static) {
Label is_nullable;
__ test_field_is_not_null_free_inline_type(flags, noreg /* temp */, is_nullable);
__ null_check(r0); // FIXME JDK-8341120
__ bind(is_nullable);
do_oop_store(_masm, field, r0, IN_HEAP);
__ b(Done);
} else {
Label null_free_reference, is_flat, rewrite_inline;
__ test_field_is_flat(flags, noreg /* temp */, is_flat);
__ test_field_is_null_free_inline_type(flags, noreg /* temp */, null_free_reference);
pop_and_check_object(obj);
// Store into the field
// Clobbers: r10, r11, r3
do_oop_store(_masm, field, r0, IN_HEAP);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_aputfield, bc, r19, true, byte_no);
}
__ b(Done);
// Implementation of the inline type semantic
__ bind(null_free_reference);
__ null_check(r0); // FIXME JDK-8341120
pop_and_check_object(obj);
// Store into the field
// Clobbers: r10, r11, r3
do_oop_store(_masm, field, r0, IN_HEAP);
__ b(rewrite_inline);
__ bind(is_flat);
pop_and_check_object(r7);
__ write_flat_field(cache, off, index, flags, r7);
__ bind(rewrite_inline);
if (rc == may_rewrite) {
patch_bytecode(Bytecodes::_fast_vputfield, bc, r19, true, byte_no);
}
__ b(Done);
}
} // Valhalla
}
__ bind(notObj);
@ -3007,7 +3176,7 @@ void TemplateTable::putfield_or_static(int byte_no, bool is_static, RewriteContr
{
Label notVolatile;
__ tbz(r5, ResolvedFieldEntry::is_volatile_shift, notVolatile);
__ tbz(flags, ResolvedFieldEntry::is_volatile_shift, notVolatile);
__ membar(MacroAssembler::StoreLoad | MacroAssembler::StoreStore);
__ bind(notVolatile);
}
@ -3041,6 +3210,7 @@ void TemplateTable::jvmti_post_fast_field_mod() {
// to do it for every data type, we use the saved values as the
// jvalue object.
switch (bytecode()) { // load values into the jvalue object
case Bytecodes::_fast_vputfield: // fall through
case Bytecodes::_fast_aputfield: __ push_ptr(r0); break;
case Bytecodes::_fast_bputfield: // fall through
case Bytecodes::_fast_zputfield: // fall through
@ -3067,6 +3237,7 @@ void TemplateTable::jvmti_post_fast_field_mod() {
r19, c_rarg2, c_rarg3);
switch (bytecode()) { // restore tos values
case Bytecodes::_fast_vputfield: // fall through
case Bytecodes::_fast_aputfield: __ pop_ptr(r0); break;
case Bytecodes::_fast_bputfield: // fall through
case Bytecodes::_fast_zputfield: // fall through
@ -3114,6 +3285,22 @@ void TemplateTable::fast_storefield(TosState state)
// access field
switch (bytecode()) {
case Bytecodes::_fast_vputfield:
{
Label is_flat, done;
__ test_field_is_flat(r5, noreg /* temp */, is_flat);
__ null_check(r0);
do_oop_store(_masm, field, r0, IN_HEAP);
__ b(done);
__ bind(is_flat);
__ load_field_entry(r4, r5);
// Re-shuffle registers because of VM calls calling convention
__ mov(r19, r1);
__ mov(r7, r2);
__ write_flat_field(r4, r19, r6, r8, r7);
__ bind(done);
}
break;
case Bytecodes::_fast_aputfield:
// Clobbers: r10, r11, r3
do_oop_store(_masm, field, r0, IN_HEAP);
@ -3209,6 +3396,13 @@ void TemplateTable::fast_accessfield(TosState state)
// access field
switch (bytecode()) {
case Bytecodes::_fast_vgetfield:
{
// field is flat
__ read_flat_field(r2, r0);
__ verify_oop(r0);
}
break;
case Bytecodes::_fast_agetfield:
do_oop_load(_masm, field, r0, IN_HEAP);
__ verify_oop(r0);
@ -3338,8 +3532,8 @@ void TemplateTable::invokevirtual_helper(Register index,
Register recv,
Register flags)
{
// Uses temporary registers r0, r3
assert_different_registers(index, recv, r0, r3);
// Uses temporary registers r0, r3, rscratch1
assert_different_registers(index, recv, r0, r3, rscratch1);
// Test for an invoke of a final method
Label notFinal;
__ tbz(flags, ResolvedMethodEntry::is_vfinal_shift, notFinal);
@ -3363,7 +3557,7 @@ void TemplateTable::invokevirtual_helper(Register index,
__ bind(notFinal);
// get receiver klass
__ load_klass(r0, recv);
__ load_klass(r0, recv, rscratch1);
// profile this call
__ profile_virtual_call(r0, rlocals);
@ -3464,7 +3658,7 @@ void TemplateTable::invokeinterface(int byte_no) {
__ tbz(r3, ResolvedMethodEntry::is_vfinal_shift, notVFinal);
// Get receiver klass into r3
__ load_klass(r3, r2);
__ load_klass(r3, r2, rscratch1);
Label subtype;
__ check_klass_subtype(r3, r0, r4, subtype);
@ -3479,7 +3673,7 @@ void TemplateTable::invokeinterface(int byte_no) {
__ bind(notVFinal);
// Get receiver klass into r3
__ load_klass(r3, r2);
__ load_klass(r3, r2, rscratch1);
Label no_such_method;
@ -3659,6 +3853,16 @@ void TemplateTable::_new() {
__ sub(r3, r3, header_size);
__ cbz(r3, initialize_header);
#ifdef ASSERT
// make sure instance_size was multiple of 8
Label L;
__ tst(r3, 7);
__ br(Assembler::EQ, L);
__ stop("object size is not multiple of 8 - adjust this code");
__ bind(L);
// must be > 0, no extra check needed here
#endif
// Initialize object fields
{
__ add(r2, r0, header_size);
@ -3671,14 +3875,16 @@ void TemplateTable::_new() {
// initialize object header only.
__ bind(initialize_header);
if (UseCompactObjectHeaders) {
if (UseCompactObjectHeaders || Arguments::is_valhalla_enabled()) {
__ ldr(rscratch1, Address(r4, Klass::prototype_header_offset()));
__ str(rscratch1, Address(r0, oopDesc::mark_offset_in_bytes()));
} else {
__ mov(rscratch1, (intptr_t)markWord::prototype().value());
__ str(rscratch1, Address(r0, oopDesc::mark_offset_in_bytes()));
}
if (!UseCompactObjectHeaders) {
__ store_klass_gap(r0, zr); // zero klass gap for compressed oops
__ store_klass(r0, r4); // store klass last
__ store_klass(r0, r4, rscratch1); // store klass last
}
if (DTraceAllocProbes) {
@ -3759,7 +3965,7 @@ void TemplateTable::checkcast()
__ load_resolved_klass_at_offset(r2, r19, r0, rscratch1); // r0 = klass
__ bind(resolved);
__ load_klass(r19, r3);
__ load_klass(r19, r3, rscratch1);
// Generate subtype check. Blows r2, r5. Object in r3.
// Superklass in r0. Subklass in r19.
@ -3774,13 +3980,12 @@ void TemplateTable::checkcast()
__ bind(ok_is_subtype);
__ mov(r0, r3); // Restore object in r3
__ b(done);
__ bind(is_null);
// Collect counts on whether this test sees nulls a lot or not.
if (ProfileInterpreter) {
__ b(done);
__ bind(is_null);
__ profile_null_seen(r2);
} else {
__ bind(is_null); // same as 'done'
}
__ bind(done);
}
@ -3805,12 +4010,12 @@ void TemplateTable::instanceof() {
__ get_vm_result_metadata(r0, rthread);
__ pop(r3); // restore receiver
__ verify_oop(r3);
__ load_klass(r3, r3);
__ load_klass(r3, r3, rscratch1);
__ b(resolved);
// Get superklass in r0 and subklass in r3
__ bind(quicked);
__ load_klass(r3, r0);
__ load_klass(r3, r0, rscratch1);
__ load_resolved_klass_at_offset(r2, r19, r0, rscratch1);
__ bind(resolved);
@ -3899,6 +4104,10 @@ void TemplateTable::monitorenter()
// check for null object
__ null_check(r0);
Label is_inline_type;
__ ldr(rscratch1, Address(r0, oopDesc::mark_offset_in_bytes()));
__ test_markword_is_inline_type(rscratch1, is_inline_type);
const Address monitor_block_top(
rfp, frame::interpreter_frame_monitor_block_top_offset * wordSize);
const Address monitor_block_bot(
@ -4000,6 +4209,11 @@ void TemplateTable::monitorenter()
// The bcp has already been incremented. Just need to dispatch to
// next instruction.
__ dispatch_next(vtos);
__ bind(is_inline_type);
__ call_VM(noreg, CAST_FROM_FN_PTR(address,
InterpreterRuntime::throw_identity_exception), r0);
__ should_not_reach_here();
}
@ -4010,6 +4224,18 @@ void TemplateTable::monitorexit()
// check for null object
__ null_check(r0);
const int is_inline_type_mask = markWord::inline_type_pattern;
Label has_identity;
__ ldr(rscratch1, Address(r0, oopDesc::mark_offset_in_bytes()));
__ mov(rscratch2, is_inline_type_mask);
__ andr(rscratch1, rscratch1, rscratch2);
__ cmp(rscratch1, rscratch2);
__ br(Assembler::NE, has_identity);
__ call_VM(noreg, CAST_FROM_FN_PTR(address,
InterpreterRuntime::throw_illegal_monitor_state_exception));
__ should_not_reach_here();
__ bind(has_identity);
const Address monitor_block_top(
rfp, frame::interpreter_frame_monitor_block_top_offset * wordSize);
const Address monitor_block_bot(

2
src/hotspot/cpu/aarch64/templateTable_aarch64.hpp
View File

@ -34,4 +34,6 @@
static void index_check(Register array, Register index);
static void index_check_without_pop(Register array, Register index);
static void invoke_is_substitutable(Register aobj, Register bobj, Label& is_subst, Label& not_subst);
#endif // CPU_AARCH64_TEMPLATETABLE_AARCH64_HPP

14
src/hotspot/cpu/aarch64/vm_version_aarch64.cpp
View File

@ -544,6 +544,13 @@ void VM_Version::initialize() {
UsePopCountInstruction = true;
}
if (supports_paca()) {
// Determine the mask of address bits used for PAC. Clear bit 55 of
// the input to make it look like a user address.
// This mask would be used in mixed jstack in SA.
_pac_mask = (uintptr_t)pauth_strip_pointer((address)~(UINT64_C(1) << 55));
}
if (UseBranchProtection == nullptr || strcmp(UseBranchProtection, "none") == 0) {
_rop_protection = false;
} else if (strcmp(UseBranchProtection, "standard") == 0 ||
@ -565,13 +572,6 @@ void VM_Version::initialize() {
} else {
vm_exit_during_initialization(err_msg("Unsupported UseBranchProtection: %s", UseBranchProtection));
}
if (_rop_protection == true) {
// Determine the mask of address bits used for PAC. Clear bit 55 of
// the input to make it look like a user address.
_pac_mask = (uintptr_t)pauth_strip_pointer((address)~(UINT64_C(1) << 55));
}
#ifdef COMPILER2
if (FLAG_IS_DEFAULT(UseMultiplyToLenIntrinsic)) {
UseMultiplyToLenIntrinsic = true;

24
src/hotspot/cpu/aarch64/vtableStubs_aarch64.cpp
View File

@ -47,10 +47,10 @@
extern "C" void bad_compiled_vtable_index(JavaThread* thread, oop receiver, int index);
#endif
VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
VtableStub* VtableStubs::create_vtable_stub(int vtable_index, bool caller_is_c1) {
// Read "A word on VtableStub sizing" in share/code/vtableStubs.hpp for details on stub sizing.
const int stub_code_length = code_size_limit(true);
VtableStub* s = new(stub_code_length) VtableStub(true, vtable_index);
VtableStub* s = new(stub_code_length) VtableStub(true, vtable_index, caller_is_c1);
// Can be null if there is no free space in the code cache.
if (s == nullptr) {
return nullptr;
@ -63,6 +63,8 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
int slop_bytes = 0;
int slop_delta = 0;
ByteSize entry_offset = caller_is_c1 ? Method::from_compiled_inline_offset() : Method::from_compiled_inline_ro_offset();
ResourceMark rm;
CodeBuffer cb(s->entry_point(), stub_code_length);
MacroAssembler* masm = new MacroAssembler(&cb);
@ -79,7 +81,7 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
// get receiver klass
address npe_addr = __ pc();
__ load_klass(r16, j_rarg0);
__ load_klass(r16, j_rarg0, rscratch1);
#ifndef PRODUCT
if (DebugVtables) {
@ -116,7 +118,7 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
if (DebugVtables) {
Label L;
__ cbz(rmethod, L);
__ ldr(rscratch1, Address(rmethod, Method::from_compiled_offset()));
__ ldr(rscratch1, Address(rmethod, entry_offset));
__ cbnz(rscratch1, L);
__ stop("Vtable entry is null");
__ bind(L);
@ -127,7 +129,7 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
// rmethod: Method*
// r2: receiver
address ame_addr = __ pc();
__ ldr(rscratch1, Address(rmethod, Method::from_compiled_offset()));
__ ldr(rscratch1, Address(rmethod, entry_offset));
__ br(rscratch1);
masm->flush();
@ -137,10 +139,10 @@ VtableStub* VtableStubs::create_vtable_stub(int vtable_index) {
}
VtableStub* VtableStubs::create_itable_stub(int itable_index) {
VtableStub* VtableStubs::create_itable_stub(int itable_index, bool caller_is_c1) {
// Read "A word on VtableStub sizing" in share/code/vtableStubs.hpp for details on stub sizing.
const int stub_code_length = code_size_limit(false);
VtableStub* s = new(stub_code_length) VtableStub(false, itable_index);
VtableStub* s = new(stub_code_length) VtableStub(false, itable_index, caller_is_c1);
// Can be null if there is no free space in the code cache.
if (s == nullptr) {
return nullptr;
@ -153,6 +155,8 @@ VtableStub* VtableStubs::create_itable_stub(int itable_index) {
int slop_bytes = 0;
int slop_delta = 0;
ByteSize entry_offset = caller_is_c1 ? Method::from_compiled_inline_offset() : Method::from_compiled_inline_ro_offset();
ResourceMark rm;
CodeBuffer cb(s->entry_point(), stub_code_length);
MacroAssembler* masm = new MacroAssembler(&cb);
@ -189,7 +193,7 @@ VtableStub* VtableStubs::create_itable_stub(int itable_index) {
// get receiver klass (also an implicit null-check)
address npe_addr = __ pc();
__ load_klass(recv_klass_reg, j_rarg0);
__ load_klass(recv_klass_reg, j_rarg0, rscratch1);
// Receiver subtype check against REFC.
// Get selected method from declaring class and itable index
@ -207,7 +211,7 @@ VtableStub* VtableStubs::create_itable_stub(int itable_index) {
if (DebugVtables) {
Label L2;
__ cbz(rmethod, L2);
__ ldr(rscratch1, Address(rmethod, Method::from_compiled_offset()));
__ ldr(rscratch1, Address(rmethod, entry_offset));
__ cbnz(rscratch1, L2);
__ stop("compiler entrypoint is null");
__ bind(L2);
@ -217,7 +221,7 @@ VtableStub* VtableStubs::create_itable_stub(int itable_index) {
// rmethod: Method*
// j_rarg0: receiver
address ame_addr = __ pc();
__ ldr(rscratch1, Address(rmethod, Method::from_compiled_offset()));
__ ldr(rscratch1, Address(rmethod, entry_offset));
__ br(rscratch1);
__ bind(L_no_such_interface);

41
src/hotspot/cpu/arm/arm.ad
View File

@ -339,10 +339,6 @@ void MachPrologNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
}
}
uint MachPrologNode::size(PhaseRegAlloc *ra_) const {
return MachNode::size(ra_);
}
int MachPrologNode::reloc() const {
return 10; // a large enough number
}
@ -384,10 +380,6 @@ void MachEpilogNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
}
}
uint MachEpilogNode::size(PhaseRegAlloc *ra_) const {
return MachNode::size(ra_);
}
int MachEpilogNode::reloc() const {
return 16; // a large enough number
}
@ -849,6 +841,18 @@ uint BoxLockNode::size(PhaseRegAlloc *ra_) const {
}
//=============================================================================
#ifndef PRODUCT
void MachVEPNode::format(PhaseRegAlloc* ra_, outputStream* st) const
{
Unimplemented();
}
#endif
void MachVEPNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc* ra_) const
{
Unimplemented();
}
#ifndef PRODUCT
#define R_RTEMP "R_R12"
void MachUEPNode::format( PhaseRegAlloc *ra_, outputStream *st ) const {
@ -863,11 +867,6 @@ void MachUEPNode::emit(C2_MacroAssembler *masm, PhaseRegAlloc *ra_) const {
__ ic_check(InteriorEntryAlignment);
}
uint MachUEPNode::size(PhaseRegAlloc *ra_) const {
return MachNode::size(ra_);
}
//=============================================================================
int HandlerImpl::emit_deopt_handler(C2_MacroAssembler* masm) {
@ -1112,26 +1111,26 @@ uint Matcher::float_pressure_limit()
return (FLOATPRESSURE == -1) ? 30 : FLOATPRESSURE;
}
// Register for DIVI projection of divmodI
const RegMask& Matcher::divI_proj_mask() {
// Register for the first projection of an int pair
const RegMask& Matcher::firstI_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for MODI projection of divmodI
const RegMask& Matcher::modI_proj_mask() {
// Register for the second projection of an int pair
const RegMask& Matcher::secondI_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for DIVL projection of divmodL
const RegMask& Matcher::divL_proj_mask() {
// Register for the first projection of a long pair
const RegMask& Matcher::firstL_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}
// Register for MODL projection of divmodL
const RegMask& Matcher::modL_proj_mask() {
// Register for the second projection of a long pair
const RegMask& Matcher::secondL_proj_mask() {
ShouldNotReachHere();
return RegMask::EMPTY;
}

6
src/hotspot/cpu/arm/arm_32.ad
View File

@ -430,18 +430,18 @@ OptoRegPair c2::return_value(int ideal_reg) {
// from the start of the call to the point where the return address
// will point.
int MachCallStaticJavaNode::ret_addr_offset() {
int MachCallStaticJavaNode::ret_addr_offset() const {
bool far = (_method == nullptr) ? maybe_far_call(this) : !cache_reachable();
return (far ? 3 : 1) * NativeInstruction::instruction_size;
}
int MachCallDynamicJavaNode::ret_addr_offset() {
int MachCallDynamicJavaNode::ret_addr_offset() const {
bool far = !cache_reachable();
// mov_oop is always 2 words
return (2 + (far ? 3 : 1)) * NativeInstruction::instruction_size;
}
int MachCallRuntimeNode::ret_addr_offset() {
int MachCallRuntimeNode::ret_addr_offset() const {
// bl or movw; movt; blx
bool far = maybe_far_call(this);
return (far ? 3 : 1) * NativeInstruction::instruction_size;

33
src/hotspot/cpu/arm/c1_CodeStubs_arm.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2008, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2008, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -156,11 +156,13 @@ void NewTypeArrayStub::emit_code(LIR_Assembler* ce) {
// Implementation of NewObjectArrayStub
NewObjectArrayStub::NewObjectArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result, CodeEmitInfo* info) {
NewObjectArrayStub::NewObjectArrayStub(LIR_Opr klass_reg, LIR_Opr length, LIR_Opr result,
CodeEmitInfo* info, bool is_null_free) {
_klass_reg = klass_reg;
_result = result;
_length = length;
_info = new CodeEmitInfo(info);
_is_null_free = is_null_free; // unimplemented
}
@ -424,4 +426,31 @@ void ArrayCopyStub::emit_code(LIR_Assembler* ce) {
__ b(_continuation);
}
// Implementation of SubstitutabilityCheckStub
SubstitutabilityCheckStub::SubstitutabilityCheckStub(LIR_Opr left, LIR_Opr right, CodeEmitInfo* info) {
Unimplemented();
}
void SubstitutabilityCheckStub::emit_code(LIR_Assembler* ce) {
Unimplemented();
}
LoadFlattenedArrayStub::LoadFlattenedArrayStub(LIR_Opr array, LIR_Opr index, LIR_Opr result, CodeEmitInfo* info) {
Unimplemented();
}
void LoadFlattenedArrayStub::emit_code(LIR_Assembler* ce) {
Unimplemented();
}
// Implementation of StoreFlattenedArrayStub
StoreFlattenedArrayStub::StoreFlattenedArrayStub(LIR_Opr array, LIR_Opr index, LIR_Opr value, CodeEmitInfo* info) {
Unimplemented();
}
void StoreFlattenedArrayStub::emit_code(LIR_Assembler* ce) {
Unimplemented();
}
#undef __

29
src/hotspot/cpu/arm/c1_LIRAssembler_arm.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2008, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2008, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -2559,6 +2559,10 @@ void LIR_Assembler::emit_profile_type(LIR_OpProfileType* op) {
fatal("Type profiling not implemented on this platform");
}
void LIR_Assembler::emit_profile_inline_type(LIR_OpProfileInlineType* op) {
Unimplemented();
}
void LIR_Assembler::monitor_address(int monitor_no, LIR_Opr dst) {
Address mon_addr = frame_map()->address_for_monitor_lock(monitor_no);
__ add_slow(dst->as_pointer_register(), mon_addr.base(), mon_addr.disp());
@ -2858,4 +2862,27 @@ void LIR_Assembler::atomic_op(LIR_Code code, LIR_Opr src, LIR_Opr data, LIR_Opr
}
// Valhalla support
void LIR_Assembler::check_orig_pc() {
Unimplemented();
}
int LIR_Assembler::store_inline_type_fields_to_buf(ciInlineKlass* vk) {
Unimplemented();
return 0;
}
void LIR_Assembler::emit_opFlattenedArrayCheck(LIR_OpFlattenedArrayCheck* op) {
Unimplemented();
}
void LIR_Assembler::emit_opNullFreeArrayCheck(LIR_OpNullFreeArrayCheck* op) {
Unimplemented();
}
void LIR_Assembler::emit_opSubstitutabilityCheck(LIR_OpSubstitutabilityCheck* op) {
Unimplemented();
}
#undef __

7
src/hotspot/cpu/arm/c1_LIRGenerator_arm.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2008, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2008, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -944,7 +944,7 @@ void LIRGenerator::do_NewInstance(NewInstance* x) {
LIR_Opr tmp2 = new_register(objectType);
LIR_Opr tmp3 = FrameMap::LR_oop_opr;
new_instance(reg, x->klass(), x->is_unresolved(), tmp1, tmp2, tmp3,
new_instance(reg, x->klass(), x->is_unresolved(), /* allow_inline */ false, tmp1, tmp2, tmp3,
LIR_OprFact::illegalOpr, klass_reg, info);
LIR_Opr result = rlock_result(x);
@ -1104,7 +1104,8 @@ void LIRGenerator::do_CheckCast(CheckCast* x) {
LIR_Opr tmp3 = LIR_OprFact::illegalOpr;
__ checkcast(out_reg, obj.result(), x->klass(), tmp1, tmp2, tmp3, x->direct_compare(),
info_for_exception, patching_info, stub, x->profiled_method(), x->profiled_bci());
info_for_exception, patching_info, stub,
x->profiled_method(), x->profiled_bci(), /*is_null_free*/ false);
}

16
src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2008, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2008, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -42,7 +42,10 @@
// arm [macro]assembler) and used with care in the other C1 specific
// files.
void C1_MacroAssembler::build_frame(int frame_size_in_bytes, int bang_size_in_bytes) {
void C1_MacroAssembler::build_frame(int frame_size_in_bytes, int bang_size_in_bytes,
int sp_offset_for_orig_pc,
bool needs_stack_repair, bool has_scalarized_args,
Label* verified_inline_entry_label) {
assert(bang_size_in_bytes >= frame_size_in_bytes, "stack bang size incorrect");
assert((frame_size_in_bytes % StackAlignmentInBytes) == 0, "frame size should be aligned");
@ -59,11 +62,6 @@ void C1_MacroAssembler::build_frame(int frame_size_in_bytes, int bang_size_in_by
bs->nmethod_entry_barrier(this);
}
void C1_MacroAssembler::remove_frame(int frame_size_in_bytes) {
add_slow(SP, SP, frame_size_in_bytes);
raw_pop(FP, LR);
}
void C1_MacroAssembler::verified_entry(bool breakAtEntry) {
if (breakAtEntry) {
breakpoint();
@ -238,4 +236,8 @@ void C1_MacroAssembler::verify_not_null_oop(Register r) {
verify_oop(r);
}
int C1_MacroAssembler::scalarized_entry(const CompiledEntrySignature* ces, int frame_size_in_bytes, int bang_size_in_bytes, int sp_offset_for_orig_pc, Label& verified_inline_entry_label, bool is_inline_ro_entry) {
Unimplemented();
}
#endif // !PRODUCT

8
src/hotspot/cpu/arm/continuationFreezeThaw_arm.inline.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2019, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2019, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -39,7 +39,7 @@ inline frame FreezeBase::sender(const frame& f) {
return frame();
}
template<typename FKind> frame FreezeBase::new_heap_frame(frame& f, frame& caller) {
template<typename FKind> frame FreezeBase::new_heap_frame(frame& f, frame& caller, int size_adjust) {
Unimplemented();
return frame();
}
@ -56,7 +56,7 @@ inline void FreezeBase::relativize_interpreted_frame_metadata(const frame& f, co
Unimplemented();
}
inline void FreezeBase::patch_pd(frame& hf, const frame& caller) {
inline void FreezeBase::patch_pd(frame& hf, const frame& caller, bool is_bottom_frame) {
Unimplemented();
}
@ -82,7 +82,7 @@ inline frame ThawBase::new_entry_frame() {
return frame();
}
template<typename FKind> frame ThawBase::new_stack_frame(const frame& hf, frame& caller, bool bottom) {
template<typename FKind> frame ThawBase::new_stack_frame(const frame& hf, frame& caller, bool bottom, int size_adjust) {
Unimplemented();
return frame();
}

15
src/hotspot/cpu/arm/frame_arm.cpp
View File

@ -510,3 +510,18 @@ intptr_t* frame::real_fp() const {
assert(! is_compiled_frame(), "unknown compiled frame size");
return fp();
}
intptr_t* frame::repair_sender_sp(intptr_t* sender_sp, intptr_t** saved_fp_addr) const {
Unimplemented();
return nullptr;
}
intptr_t* frame::repair_sender_sp(nmethod* nm, intptr_t* sp, intptr_t** saved_fp_addr) {
Unimplemented();
return nullptr;
}
bool frame::was_augmented_on_entry(int& real_size) const {
Unimplemented();
return false;
}

Some files were not shown because too many files have changed in this diff Show More