infernap12/jdk
1
0
Fork 0
mirror of https://github.com/openjdk/jdk.git synced 2026-06-10 04:25:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into JDK-8382052-bitwise-blend

Browse Source
This commit is contained in:
erfang 2026-06-09 08:50:50 +00:00
commit 0d38271f7d
358 changed files with 26794 additions and 10691 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.jcheck/conf
View File

@ -1,7 +1,7 @@
[general]
project=jdk
jbs=JDK
version=27
version=28
[checks]
error=author,committer,reviewers,merge,issues,executable,symlink,message,hg-tag,whitespace,problemlists,copyright

28
doc/building.html
View File

@ -394,11 +394,11 @@ to date at the time of writing.</p>
<tbody>
<tr class="odd">
<td>Linux/x64</td>
<td>Oracle Enterprise Linux 6.4 / 8.x</td>
<td>Oracle Linux 6.4 / 8.x</td>
</tr>
<tr class="even">
<td>Linux/aarch64</td>
<td>Oracle Enterprise Linux 7.6 / 8.x</td>
<td>Oracle Linux 7.6 / 8.x</td>
</tr>
<tr class="odd">
<td>macOS</td>
@ -1495,26 +1495,24 @@ following targets are known to work:</p>
</tr>
</tbody>
</table>
<p><code>BASE_OS</code> must be one of <code>OL</code> for Oracle
Enterprise Linux or <code>Fedora</code>. If the base OS is
<code>Fedora</code> the corresponding Fedora release can be specified
with the help of the <code>BASE_OS_VERSION</code> option. If the build
is successful, the new devkits can be found in the
<p><code>BASE_OS</code> must be one of <code>OL</code> for Oracle Linux
or <code>Fedora</code>. The release/version of the base OS can be
specified using the <code>BASE_OS_VERSION</code> option. If the build is
successful, the new devkits can be found in the
<code>build/devkit/result</code> subdirectory:</p>
<pre><code>cd make/devkit
make TARGETS=&quot;ppc64le-linux-gnu aarch64-linux-gnu&quot; BASE_OS=Fedora BASE_OS_VERSION=21
make TARGETS=&quot;ppc64le-linux-gnu aarch64-linux-gnu&quot; BASE_OS=Fedora
ls -1 ../../build/devkit/result/
x86_64-linux-gnu-to-aarch64-linux-gnu
x86_64-linux-gnu-to-ppc64le-linux-gnu</code></pre>
<p>Notice that devkits are not only useful for targeting different build
platforms. Because they contain the full build dependencies for a system
(i.e. compiler and root file system), they can easily be used to build
well-known, reliable and reproducible build environments. You can for
example create and use a devkit with GCC 7.3 and a Fedora 12 sysroot
environment (with glibc 2.11) on Ubuntu 14.04 (which doesn't have GCC
7.3 by default) to produce JDK binaries which will run on all Linux
systems with runtime libraries newer than the ones from Fedora 12 (e.g.
Ubuntu 16.04, SLES 11 or RHEL 6).</p>
(i.e., compiler and root file system/sysroot), they can easily be used
to build well-known, reliable, and reproducible build environments. You
can, for example, create and use a devkit with a version of the GCC
compiler not provided by the host OS, using a sysroot from an older
Linux distribution to produce JDK binaries which will run on all Linux
systems with newer runtime libraries.</p>
<h4 id="using-debian-debootstrap">Using Debian debootstrap</h4>
<p>On Debian (or a derivative like Ubuntu), you can create sysroots for
foreign architectures with tools provided by the OS. You can use

27
doc/building.md
View File

@ -195,8 +195,8 @@ time of writing.
| Operating system | Vendor/version used |
| ----------------- | ---------------------------------- |
| Linux/x64 | Oracle Enterprise Linux 6.4 / 8.x |
| Linux/aarch64 | Oracle Enterprise Linux 7.6 / 8.x |
| Linux/x64 | Oracle Linux 6.4 / 8.x |
| Linux/aarch64 | Oracle Linux 7.6 / 8.x |
| macOS | macOS 14.x |
| Windows | Windows Server 2016 |
@ -1288,27 +1288,26 @@ at least the following targets are known to work:
| riscv64-linux-gnu |
| s390x-linux-gnu |
`BASE_OS` must be one of `OL` for Oracle Enterprise Linux or `Fedora`. If the
base OS is `Fedora` the corresponding Fedora release can be specified with the
help of the `BASE_OS_VERSION` option. If the build is successful, the new
devkits can be found in the `build/devkit/result` subdirectory:
`BASE_OS` must be one of `OL` for Oracle Linux or `Fedora`. The release/version
of the base OS can be specified using the `BASE_OS_VERSION` option. If the build
is successful, the new devkits can be found in the `build/devkit/result`
subdirectory:
```
cd make/devkit
make TARGETS="ppc64le-linux-gnu aarch64-linux-gnu" BASE_OS=Fedora BASE_OS_VERSION=21
make TARGETS="ppc64le-linux-gnu aarch64-linux-gnu" BASE_OS=Fedora
ls -1 ../../build/devkit/result/
x86_64-linux-gnu-to-aarch64-linux-gnu
x86_64-linux-gnu-to-ppc64le-linux-gnu
```
Notice that devkits are not only useful for targeting different build
platforms. Because they contain the full build dependencies for a system (i.e.
compiler and root file system), they can easily be used to build well-known,
reliable and reproducible build environments. You can for example create and
use a devkit with GCC 7.3 and a Fedora 12 sysroot environment (with glibc 2.11)
on Ubuntu 14.04 (which doesn't have GCC 7.3 by default) to produce JDK binaries
which will run on all Linux systems with runtime libraries newer than the ones
from Fedora 12 (e.g. Ubuntu 16.04, SLES 11 or RHEL 6).
platforms. Because they contain the full build dependencies for a system (i.e.,
compiler and root file system/sysroot), they can easily be used to build
well-known, reliable, and reproducible build environments. You can, for example,
create and use a devkit with a version of the GCC compiler not provided by the
host OS, using a sysroot from an older Linux distribution to produce JDK
binaries which will run on all Linux systems with newer runtime libraries.
#### Using Debian debootstrap

18
make/CompileDemos.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -173,41 +173,41 @@ $(BUILD_DEMO_CodePointIM_JAR): $(CODEPOINT_METAINF_SERVICE_FILE)
$(eval $(call SetupBuildDemo, FileChooserDemo, \
DEMO_SUBDIR := jfc, \
DISABLED_WARNINGS := rawtypes deprecation unchecked this-escape, \
DISABLED_WARNINGS := this-escape, \
))
$(eval $(call SetupBuildDemo, SwingSet2, \
DEMO_SUBDIR := jfc, \
EXTRA_COPY_TO_JAR := .java, \
EXTRA_MANIFEST_ATTR := SplashScreen-Image: resources/images/splash.png, \
DISABLED_WARNINGS := rawtypes deprecation unchecked static serial cast this-escape, \
DISABLED_WARNINGS := rawtypes static serial cast this-escape, \
))
$(eval $(call SetupBuildDemo, Font2DTest, \
DISABLED_WARNINGS := rawtypes deprecation unchecked serial cast this-escape dangling-doc-comments, \
DISABLED_WARNINGS := serial dangling-doc-comments, \
DEMO_SUBDIR := jfc, \
))
$(eval $(call SetupBuildDemo, J2Ddemo, \
DEMO_SUBDIR := jfc, \
MAIN_CLASS := java2d.J2Ddemo, \
DISABLED_WARNINGS := rawtypes deprecation unchecked cast lossy-conversions this-escape, \
DISABLED_WARNINGS := cast lossy-conversions this-escape, \
JAR_NAME := J2Ddemo, \
))
$(eval $(call SetupBuildDemo, Metalworks, \
DISABLED_WARNINGS := rawtypes unchecked this-escape, \
DISABLED_WARNINGS := this-escape, \
DEMO_SUBDIR := jfc, \
))
$(eval $(call SetupBuildDemo, Notepad, \
DISABLED_WARNINGS := rawtypes this-escape, \
DISABLED_WARNINGS := this-escape, \
DEMO_SUBDIR := jfc, \
))
$(eval $(call SetupBuildDemo, Stylepad, \
DEMO_SUBDIR := jfc, \
DISABLED_WARNINGS := rawtypes unchecked this-escape, \
DISABLED_WARNINGS := this-escape, \
EXTRA_SRC_DIR := $(DEMO_SHARE_SRC)/jfc/Notepad, \
EXCLUDE_FILES := $(DEMO_SHARE_SRC)/jfc/Notepad/README.txt, \
))
@ -217,7 +217,7 @@ $(eval $(call SetupBuildDemo, SampleTree, \
))
$(eval $(call SetupBuildDemo, TableExample, \
DISABLED_WARNINGS := rawtypes unchecked deprecation this-escape dangling-doc-comments, \
DISABLED_WARNINGS := dangling-doc-comments, \
DEMO_SUBDIR := jfc, \
))

4
make/conf/jib-profiles.js
View File

@ -1192,8 +1192,8 @@ var getJibProfilesDependencies = function (input, common) {
server: "jpg",
product: "jcov",
version: "3.0",
build_number: "5",
file: "bundles/jcov-3.0+5.zip",
build_number: "6",
file: "bundles/jcov-3.0+6.zip",
environment_name: "JCOV_HOME",
},

10
make/conf/version-numbers.conf
View File

@ -26,17 +26,17 @@
# Default version, product, and vendor information to use,
# unless overridden by configure
DEFAULT_VERSION_FEATURE=27
DEFAULT_VERSION_FEATURE=28
DEFAULT_VERSION_INTERIM=0
DEFAULT_VERSION_UPDATE=0
DEFAULT_VERSION_PATCH=0
DEFAULT_VERSION_EXTRA1=0
DEFAULT_VERSION_EXTRA2=0
DEFAULT_VERSION_EXTRA3=0
DEFAULT_VERSION_DATE=2026-09-15
DEFAULT_VERSION_CLASSFILE_MAJOR=71 # "`$EXPR $DEFAULT_VERSION_FEATURE + 44`"
DEFAULT_VERSION_DATE=2027-03-23
DEFAULT_VERSION_CLASSFILE_MAJOR=72 # "`$EXPR $DEFAULT_VERSION_FEATURE + 44`"
DEFAULT_VERSION_CLASSFILE_MINOR=0
DEFAULT_VERSION_DOCS_API_SINCE=11
DEFAULT_ACCEPTABLE_BOOT_VERSIONS="26 27"
DEFAULT_JDK_SOURCE_TARGET_VERSION=27
DEFAULT_ACCEPTABLE_BOOT_VERSIONS="26 27 28"
DEFAULT_JDK_SOURCE_TARGET_VERSION=28
DEFAULT_PROMOTED_VERSION_PRE=ea

107
make/devkit/Common.gmk Normal file
View File

@ -0,0 +1,107 @@
#
# Copyright (c) 2013, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 only, as
# published by the Free Software Foundation. Oracle designates this
# particular file as subject to the "Classpath" exception as provided
# by Oracle in the LICENSE file that accompanied this code.
#
# This code is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# version 2 for more details (a copy is included in the LICENSE file that
# accompanied this code).
#
# You should have received a copy of the GNU General Public License version
# 2 along with this work; if not, write to the Free Software Foundation,
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
#
# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
# or visit www.oracle.com if you need additional information or have any
# questions.
#
ARCH := $(word 1,$(subst -, ,$(TARGET)))
ifeq ($(TARGET), arm-linux-gnueabihf)
ARCH=armhfp
endif
$(info ARCH=$(ARCH))
ifeq ($(PREFIX),)
$(error PREFIX not set)
endif
BUILDDIR := $(OUTPUT_ROOT)/$(HOST)/$(TARGET)
TARGETDIR := $(PREFIX)/$(TARGET)
SYSROOT := $(TARGETDIR)/sysroot
# Base OS information and repositories
#
# BASE_OS_REPOS is a space-separated set of repositories. Each entry
# is of the format <repo_name>,<url> where <repo_name> is a unique
# identifier across the set of repositories.
ifeq ($(BASE_OS), OL)
ifeq ($(filter aarch64 x86_64, $(ARCH)), )
$(error Only "aarch64 x86_64" architectures are supported for OL, but "$(ARCH)" was requested)
endif
BASE_OS_VERSION ?= 7
ifeq ($(BASE_OS_VERSION), 6)
ifeq ($(filter x86_64, $(ARCH)), )
$(error Only "x86_64" architectures are supported for OL6, but "$(ARCH)" was requested)
endif
BASE_OS_DESCRIPTION_VERSION := 6.4
REPO_BASE_URL := https://yum.oracle.com/repo/OracleLinux/OL6/4
BASE_OS_REPOS := base,$(REPO_BASE_URL)/base/$(ARCH)
else ifeq ($(BASE_OS_VERSION), 7)
BASE_OS_DESCRIPTION_VERSION := 7.6
REPO_BASE_URL := https://yum.oracle.com/repo/OracleLinux/OL7/6
BASE_OS_REPOS := base,$(REPO_BASE_URL)/base/$(ARCH)
else
REPO_BASE_URL := https://yum.oracle.com/repo/OracleLinux/OL$(BASE_OS_VERSION)
BASE_OS_REPOS := baseos,$(REPO_BASE_URL)/baseos/latest/$(ARCH) appstream,$(REPO_BASE_URL)/appstream/$(ARCH)
BASE_OS_DESCRIPTION_VERSION := $(BASE_OS_VERSION)
endif
BASE_OS_DESCRIPTION := OL$(BASE_OS_DESCRIPTION_VERSION)
else ifeq ($(BASE_OS), Fedora)
ifeq ($(filter aarch64 armhfp ppc64le riscv64 s390x x86_64, $(ARCH)), )
$(error Only "aarch64 armhfp ppc64le riscv64 s390x x86_64" architectures are supported for Fedora, but "$(ARCH)" was requested)
endif
ifeq ($(ARCH), riscv64)
BASE_OS_VERSION ?= 43
BASE_OS_BUILD := 6640
REPO_BASE_URL := https://riscv-koji.fedoraproject.org/repos-dist/f$(BASE_OS_VERSION)/$(BASE_OS_BUILD)/riscv64
BASE_OS_REPOS := riscv64,$(REPO_BASE_URL)
else
ifeq ($(ARCH), armhfp)
BASE_OS_VERSION ?= 36
else
BASE_OS_VERSION ?= 41
endif
ifeq ($(ARCH), armhfp)
ifneq ($(BASE_OS_VERSION), 36)
$(error Fedora 36 is the last release supporting "armhfp", but $(BASE_OS_VERSION) was requested)
endif
endif
LATEST_ARCHIVED_OS_VERSION := 41
ifeq ($(filter aarch64 x86_64 armhfp, $(ARCH)), )
FEDORA_TYPE := fedora-secondary
else
FEDORA_TYPE := fedora/linux
endif
NOT_ARCHIVED := $(shell [ $(BASE_OS_VERSION) -gt $(LATEST_ARCHIVED_OS_VERSION) ] && echo true)
ifeq ($(NOT_ARCHIVED),true)
RPM_REPO_URL := https://dl.fedoraproject.org/pub/$(FEDORA_TYPE)/releases/$(BASE_OS_VERSION)/Everything/$(ARCH)/os
else
RPM_REPO_URL := https://archives.fedoraproject.org/pub/archive/$(FEDORA_TYPE)/releases/$(BASE_OS_VERSION)/Everything/$(ARCH)/os
endif
BASE_OS_REPOS := os,$(RPM_REPO_URL)
endif
BASE_OS_DESCRIPTION := Fedora_$(BASE_OS_VERSION)
else
$(error Unknown base OS $(BASE_OS))
endif

33
make/devkit/Makefile
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2013, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2013, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -39,7 +39,7 @@
#
# make TARGETS="aarch64-linux-gnu" BASE_OS=Fedora
# or
# make TARGETS="arm-linux-gnueabihf ppc64le-linux-gnu" BASE_OS=Fedora BASE_OS_VERSION=17
# make TARGETS="aarch64-linux-gnu ppc64le-linux-gnu" BASE_OS=Fedora BASE_OS_VERSION=41
#
# to build several devkits for a specific OS version at once.
# You can find the final results under ../../build/devkit/result/<host>-to-<target>
@ -50,7 +50,7 @@
# makefile again for cross compilation. Ex:
#
# PATH=$PWD/../../build/devkit/result/x86_64-linux-gnu-to-x86_64-linux-gnu/bin:$PATH \
# make TARGETS="arm-linux-gnueabihf ppc64le-linux-gnu" BASE_OS=Fedora
# make TARGETS="aarch64-linux-gnu ppc64le-linux-gnu" BASE_OS=Fedora
#
# This is the makefile which iterates over all host and target platforms.
#
@ -79,10 +79,10 @@ TARGET_PLATFORMS := $(PLATFORMS)
$(info HOST_PLATFORMS $(HOST_PLATFORMS))
$(info TARGET_PLATFORMS $(TARGET_PLATFORMS))
all compile : $(PLATFORMS)
all compile: $(PLATFORMS)
ifeq ($(SKIP_ME), )
$(foreach p,$(filter-out $(ME),$(PLATFORMS)),$(eval $(p) : $$(ME)))
$(foreach p,$(filter-out $(ME),$(PLATFORMS)),$(eval $(p): $$(ME)))
endif
OUTPUT_ROOT = $(abspath ../../build/devkit)
@ -90,38 +90,39 @@ RESULT = $(OUTPUT_ROOT)/result
SUBMAKEVARS = HOST=$@ BUILD=$(ME) RESULT=$(RESULT) OUTPUT_ROOT=$(OUTPUT_ROOT)
$(HOST_PLATFORMS) :
$(HOST_PLATFORMS):
@echo 'Building compilers for $@'
@echo 'Targets: $(TARGET_PLATFORMS)'
for p in $(filter $@, $(TARGET_PLATFORMS)) $(filter-out $@, $(TARGET_PLATFORMS)); do \
$(MAKE) -f Tools.gmk download-rpms $(SUBMAKEVARS) \
$(MAKE) -f Sysroot.gmk sysroot $(SUBMAKEVARS) \
TARGET=$$p PREFIX=$(RESULT)/$@-to-$$p && \
$(MAKE) -f Tools.gmk all $(SUBMAKEVARS) \
TARGET=$$p PREFIX=$(RESULT)/$@-to-$$p && \
$(MAKE) -f Tools.gmk ccache $(SUBMAKEVARS) \
TARGET=$@ PREFIX=$(RESULT)/$@-to-$$p || exit 1 ; \
TARGET=$@ PREFIX=$(RESULT)/$@-to-$$p || exit 1; \
done
@echo 'All done"'
@echo 'All done'
TODAY := $(shell date +%Y%m%d)
define Mktar
$(1)-to-$(2)_tar = $$(RESULT)/sdk-$(1)-to-$(2)-$$(TODAY).tar.gz
$$($(1)-to-$(2)_tar) : PLATFORM = $(1)-to-$(2)
$$($(1)-to-$(2)_tar): PLATFORM = $(1)-to-$(2)
TARFILES += $$($(1)-to-$(2)_tar)
endef
$(foreach p,$(HOST_PLATFORMS),$(foreach t,$(TARGET_PLATFORMS),$(eval $(call Mktar,$(p),$(t)))))
tars : all $(TARFILES)
onlytars : $(TARFILES)
%.tar.gz :
tars: all $(TARFILES)
onlytars: $(TARFILES)
%.tar.gz:
$(MAKE) -r -f Tars.gmk SRC_DIR=$(RESULT)/$(PLATFORM) TAR_FILE=$@
clean :
clean:
rm -rf $(addprefix ../../build/devkit/, result $(HOST_PLATFORMS))
dist-clean: clean
rm -rf $(addprefix ../../build/devkit/, src download)
FORCE :
.PHONY : all compile tars $(HOST_PLATFORMS) clean dist-clean
.PHONY: all compile tars $(HOST_PLATFORMS) clean dist-clean

208
make/devkit/Sysroot.gmk Normal file
View File

@ -0,0 +1,208 @@
#
# Copyright (c) 2013, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 only, as
# published by the Free Software Foundation. Oracle designates this
# particular file as subject to the "Classpath" exception as provided
# by Oracle in the LICENSE file that accompanied this code.
#
# This code is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# version 2 for more details (a copy is included in the LICENSE file that
# accompanied this code).
#
# You should have received a copy of the GNU General Public License version
# 2 along with this work; if not, write to the Free Software Foundation,
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
#
# Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
# or visit www.oracle.com if you need additional information or have any
# questions.
#
include Common.gmk
$(info TARGET=$(TARGET))
$(info HOST=$(HOST))
$(info BUILD=$(BUILD))
ifeq ($(BASE_OS)-$(BASE_OS_VERSION)-$(ARCH), OL-7-aarch64)
KERNEL_HEADERS_RPM := kernel-uek-headers
else
KERNEL_HEADERS_RPM := kernel-headers
endif
ifneq ($(BASE_OS)-$(BASE_OS_VERSION), OL-6)
WAYLAND_RPMS := wayland-devel wayland-protocols-devel
endif
ZLIB_RPMS := zlib zlib-devel
ifeq ($(BASE_OS), Fedora)
ifneq ($(ARCH), armhfp)
ZLIB_RPMS := zlib-ng zlib-ng-devel
endif
endif
################################################################################
# RPMs to include
RPM_LIST := \
filesystem \
$(KERNEL_HEADERS_RPM) \
glibc glibc-devel \
cups-libs cups-devel \
libX11 libX11-devel \
libxcb xorg-x11-proto-devel \
alsa-lib alsa-lib-devel \
libXext libXext-devel \
libXtst libXtst-devel \
libXrender libXrender-devel \
libXrandr libXrandr-devel \
freetype freetype-devel \
libXt libXt-devel \
libSM libSM-devel \
libICE libICE-devel \
libXi libXi-devel \
libXau libXau-devel \
libgcc \
$(ZLIB_RPMS) \
libffi libffi-devel \
fontconfig fontconfig-devel \
systemtap-sdt-devel \
$(WAYLAND_RPMS) \
#
################################################################################
# Define common directories and files
DOWNLOAD := $(OUTPUT_ROOT)/download
DOWNLOAD_RPMS := $(DOWNLOAD)/rpms/$(TARGET)-$(BASE_OS)-$(BASE_OS_VERSION)
SRCDIR := $(OUTPUT_ROOT)/src
################################################################################
# Marker files
DOWNLOAD_RPMS_MARKER := $(BUILDDIR)/download-rpms.marker
RPMS_UNPACKED_MARKER := $(BUILDDIR)/rpms_unpacked.marker
UNPATCHED_SYSROOT_MARKER := $(BUILDDIR)/sysroot_unpatched.marker
PATCHED_SYSROOT_MARKER := $(BUILDDIR)/sysroot_patched.marker
################################################################################
# Download RPMs
ifeq ($(ARCH), armhfp)
RPM_ARCH := armv7hl
else
RPM_ARCH := $(ARCH)
endif
RPM_ARCHS := $(RPM_ARCH) noarch
ifeq ($(ARCH), x86_64)
# Enable mixed mode.
RPM_ARCHS += i386 i686
endif
EMPTY :=
SPACE := $(EMPTY) $(EMPTY)
COMMA := ,
DNF_ARCHS := $(foreach arch,$(RPM_ARCHS),--arch $(arch))
# Specify a dummy installation root, otherwise dnf will run into
# problems trying to reconcile with the local/system state
DNF_DUMMY_INSTALL_ROOT := $(BUILDDIR)/dnf-dummy-install-root
DNF_REPOS := $(foreach repo, $(BASE_OS_REPOS), \
--repofrompath $(repo) \
--enablerepo $(word 1,$(subst $(COMMA),$(SPACE),$(repo))))
DNF_DOWNLOAD_FLAGS := \
--disablerepo='*' \
$(DNF_REPOS) \
--resolve \
$(DNF_ARCHS) \
--forcearch $(RPM_ARCH) \
--installroot $(DNF_DUMMY_INSTALL_ROOT) \
--releasever $(BASE_OS_VERSION) \
#
$(DOWNLOAD_RPMS_MARKER):
@mkdir -p $(@D)
mkdir -p $(DOWNLOAD_RPMS)
echo $(RPM_LIST) | \
xargs dnf download $(DNF_DOWNLOAD_FLAGS) --destdir $(DOWNLOAD_RPMS)
touch $@
################################################################################
# Unpack RPMS
RPM_PATTERNS := $(foreach arch,$(RPM_ARCHS),$(DOWNLOAD_RPMS)/*.$(arch).rpm)
CPIO_EXCLUDES := \
"./usr/share/doc/*" \
"./usr/share/man/*" \
"./usr/X11R6/man/*" \
"*/X11/locale/*" \
#
$(RPMS_UNPACKED_MARKER): $(DOWNLOAD_RPMS_MARKER)
if [ -d $(SYSROOT) ]; then echo "WARNING: Sysroot directory ($(SYSROOT)) already exists, proceeding anyway..."; fi
@mkdir -p $(SYSROOT)
# The -e test below is needed to skip unmatched glob patterns
( \
cd $(SYSROOT); \
for rpm in $(RPM_PATTERNS); do \
if [ ! -e "$$rpm" ]; then continue; fi; \
echo Extracting $$rpm...; \
rpm2cpio $$rpm | \
cpio --extract --make-directories -f $(CPIO_EXCLUDES) \
|| exit 1; \
done \
)
touch $@
################################################################################
$(UNPATCHED_SYSROOT_MARKER): $(RPMS_UNPACKED_MARKER)
touch $@
################################################################################
# Patch sysroot
# Note: MUST create a <sysroot>/usr/lib even if not really needed.
# gcc will use a path relative to it to resolve lib64. (x86_64).
# we're creating multi-lib compiler with 32bit libc as well, so we should
# have it anyway, but just to make sure...
# Patch GNU ld scripts to force linking against libraries in the sysroot
# and not the ones installed on the build machine.
LD_SCRIPT_PATCHES := \
-e 's|/usr/lib64/||g' \
-e 's|/usr/lib/||g' \
-e 's|/lib64/||g' \
-e 's|/lib/||g' \
#
$(PATCHED_SYSROOT_MARKER): $(UNPATCHED_SYSROOT_MARKER)
@echo Patching GNU ld scripts
@( \
for f in $$(find $(SYSROOT) -name "*.so" -type f 2>/dev/null); do \
if grep -Iq 'GNU ld script' "$$f"; then \
sed $(LD_SCRIPT_PATCHES) "$$f" > "$$f.tmp" && \
mv "$$f.tmp" "$$f"; \
fi; \
done \
)
@mkdir -p $(SYSROOT)/usr/lib
@touch $@
################################################################################
download-rpms: $(DOWNLOAD_RPMS_MARKER)
unpatched-sysroot: $(UNPATCHED_SYSROOT_MARKER)
sysroot: $(PATCHED_SYSROOT_MARKER)
.PHONY: download-rpms unpatched-sysroot sysroot

6
make/devkit/Tars.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2013, 2019, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2013, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -39,9 +39,9 @@ endif
default: tars
tars : $(TAR_FILE)
tars: $(TAR_FILE)
$(TAR_FILE): $(shell find $(SRC_DIR) -type f)
$(TAR_FILE): $(shell find $(SRC_DIR) -type f | sed 's/ /\\ /g')
@echo 'Creating compiler package $@'
cd $(dir $(SRC_DIR)) && tar -czf $@ $(notdir $(SRC_DIR))/*
touch $@

362
make/devkit/Tools.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2013, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2013, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -39,68 +39,14 @@
# Fix this...
#
include Common.gmk
lowercase = $(shell echo $1 | tr A-Z a-z)
$(info TARGET=$(TARGET))
$(info HOST=$(HOST))
$(info BUILD=$(BUILD))
ARCH := $(word 1,$(subst -, ,$(TARGET)))
ifeq ($(TARGET), arm-linux-gnueabihf)
ARCH=armhfp
endif
$(info ARCH=$(ARCH))
KERNEL_HEADERS_RPM := kernel-headers
ifeq ($(BASE_OS), OL)
ifeq ($(ARCH), aarch64)
BASE_URL := https://yum.oracle.com/repo/OracleLinux/OL7/6/base/$(ARCH)/
LINUX_VERSION := OL7.6
KERNEL_HEADERS_RPM := kernel-uek-headers
else
BASE_URL := https://yum.oracle.com/repo/OracleLinux/OL6/4/base/$(ARCH)/
LINUX_VERSION := OL6.4
endif
else ifeq ($(BASE_OS), Fedora)
DEFAULT_OS_VERSION := 41
ifeq ($(BASE_OS_VERSION), )
BASE_OS_VERSION := $(DEFAULT_OS_VERSION)
endif
ifeq ($(filter aarch64 armhfp ppc64le riscv64 s390x x86_64, $(ARCH)), )
$(error Only "aarch64 armhfp ppc64le riscv64 s390x x86_64" architectures are supported for Fedora, but "$(ARCH)" was requested)
endif
ifeq ($(ARCH), riscv64)
ifeq ($(filter 38 39 40 41, $(BASE_OS_VERSION)), )
$(error Only Fedora 38-41 are supported for "$(ARCH)", but Fedora $(BASE_OS_VERSION) was requested)
endif
BASE_URL := http://fedora.riscv.rocks/repos-dist/f$(BASE_OS_VERSION)/latest/$(ARCH)/Packages/
else
LATEST_ARCHIVED_OS_VERSION := 41
ifeq ($(filter aarch64 armhfp x86_64, $(ARCH)), )
FEDORA_TYPE := fedora-secondary
else
FEDORA_TYPE := fedora/linux
endif
ifeq ($(ARCH), armhfp)
ifneq ($(BASE_OS_VERSION), 36)
$(error Fedora 36 is the last release supporting "armhfp", but $(BASE_OS) was requested)
endif
endif
NOT_ARCHIVED := $(shell [ $(BASE_OS_VERSION) -gt $(LATEST_ARCHIVED_OS_VERSION) ] && echo true)
ifeq ($(NOT_ARCHIVED),true)
BASE_URL := https://dl.fedoraproject.org/pub/$(FEDORA_TYPE)/releases/$(BASE_OS_VERSION)/Everything/$(ARCH)/os/Packages/
else
BASE_URL := https://archives.fedoraproject.org/pub/archive/$(FEDORA_TYPE)/releases/$(BASE_OS_VERSION)/Everything/$(ARCH)/os/Packages/
endif
endif
LINUX_VERSION := Fedora_$(BASE_OS_VERSION)
else
$(error Unknown base OS $(BASE_OS))
endif
################################################################################
# Define external dependencies
@ -156,32 +102,6 @@ ifneq ($(REQUIRED_MIN_MAKE_MAJOR_VERSION),)
endif
endif
# RPMs used by all BASE_OS
RPM_LIST := \
$(KERNEL_HEADERS_RPM) \
glibc glibc-headers glibc-devel \
cups-libs cups-devel \
libX11 libX11-devel \
libxcb xorg-x11-proto-devel \
alsa-lib alsa-lib-devel \
libXext libXext-devel \
libXtst libXtst-devel \
libXrender libXrender-devel \
libXrandr libXrandr-devel \
freetype freetype-devel \
libXt libXt-devel \
libSM libSM-devel \
libICE libICE-devel \
libXi libXi-devel \
libXdmcp libXdmcp-devel \
libXau libXau-devel \
libgcc libxcrypt \
zlib zlib-devel \
libffi libffi-devel \
fontconfig fontconfig-devel \
systemtap-sdt-devel \
#
################################################################################
# Define common directories and files
@ -194,28 +114,11 @@ else
endif
# Define directories
BUILDDIR := $(OUTPUT_ROOT)/$(HOST)/$(TARGET)
TARGETDIR := $(PREFIX)/$(TARGET)
SYSROOT := $(TARGETDIR)/sysroot
DOWNLOAD := $(OUTPUT_ROOT)/download
DOWNLOAD_RPMS := $(DOWNLOAD)/rpms/$(TARGET)-$(LINUX_VERSION)
SRCDIR := $(OUTPUT_ROOT)/src
# Marker file for unpacking rpms
RPMS := $(SYSROOT)/rpms_unpacked
# Need to patch libs that are linker scripts to use non-absolute paths
LIBS := $(SYSROOT)/libs_patched
################################################################################
# Download RPMs
download-rpms:
mkdir -p $(DOWNLOAD_RPMS)
# Only run this if rpm dir is empty.
ifeq ($(wildcard $(DOWNLOAD_RPMS)/*.rpm), )
cd $(DOWNLOAD_RPMS) && \
wget -r -np -nd $(patsubst %, -A "*%*.rpm", $(RPM_LIST)) $(BASE_URL)
endif
# Marker files
LINK_LIBS_MARKER := $(BUILDDIR)/link_libs.marker
################################################################################
# Unpack source packages
@ -236,24 +139,24 @@ define DownloadVerify
endif
$(1)_FILE = $(DOWNLOAD)/$(notdir $($(1)_URL))
$$($(1)_SRC_MARKER) : $$($(1)_FILE)
$$($(1)_SRC_MARKER): $$($(1)_FILE)
mkdir -p $$(SRCDIR)
tar -C $$(SRCDIR) -xf $$<
$$(foreach p,$$(abspath $$(wildcard patches/$$(ARCH)-$$(notdir $$($(1)_DIR)).patch)), \
echo PATCHING $$(p) ; \
patch -d $$($(1)_DIR) -p1 -i $$(p) ; \
echo PATCHING $$(p); \
patch -d $$($(1)_DIR) -p1 -i $$(p); \
)
touch $$@
$$($(1)_FILE) :
$$($(1)_FILE):
mkdir -p $$(@D)
wget -O - $$($(1)_URL) > $$@.tmp
sha512_actual="$$$$(sha512sum $$@.tmp | awk '{ print $$$$1; }')"; \
if [ x"$$$${sha512_actual}" != x"$$($(1)_SHA512)" ]; then \
echo "Checksum mismatch for $$@.tmp"; \
echo " Expected: $$($(1)_SHA512)"; \
echo " Actual: $$$${sha512_actual}"; \
exit 1; \
echo "Checksum mismatch for $$@.tmp"; \
echo " Expected: $$($(1)_SHA512)"; \
echo " Actual: $$$${sha512_actual}"; \
exit 1; \
fi
mv $$@.tmp $$@
endef
@ -261,89 +164,17 @@ endef
# Download and unpack all source packages
$(foreach dep,$(DEPENDENCIES),$(eval $(call DownloadVerify,$(dep))))
################################################################################
# Unpack RPMS
RPM_ARCHS := $(ARCH) noarch
ifeq ($(ARCH),x86_64)
# Enable mixed mode.
RPM_ARCHS += i386 i686
else ifeq ($(ARCH),i686)
RPM_ARCHS += i386
else ifeq ($(ARCH), armhfp)
RPM_ARCHS += armv7hl
endif
RPM_FILE_LIST := $(sort $(foreach a, $(RPM_ARCHS), \
$(wildcard $(patsubst %,$(DOWNLOAD_RPMS)/%*$a.rpm,$(RPM_LIST))) \
))
# Note. For building linux you should install rpm2cpio.
define unrpm
$(SYSROOT)/$(notdir $(1)).unpacked : $(1)
$$(RPMS) : $(SYSROOT)/$(notdir $(1)).unpacked
endef
%.unpacked :
$(info Unpacking target rpms and libraries from $<)
@(mkdir -p $(@D); \
cd $(@D); \
rpm2cpio $< | \
cpio --extract --make-directories \
-f \
"./usr/share/doc/*" \
"./usr/share/man/*" \
"./usr/X11R6/man/*" \
"*/X11/locale/*" \
|| die ; )
touch $@
$(foreach p,$(RPM_FILE_LIST),$(eval $(call unrpm,$(p))))
################################################################################
# Note: MUST create a <sysroot>/usr/lib even if not really needed.
# gcc will use a path relative to it to resolve lib64. (x86_64).
# we're creating multi-lib compiler with 32bit libc as well, so we should
# have it anyway, but just to make sure...
# Patch libc.so and libpthread.so to force linking against libraries in sysroot
# and not the ones installed on the build machine.
$(LIBS) : $(RPMS)
@echo Patching libc and pthreads
@(for f in `find $(SYSROOT) -name libc.so -o -name libpthread.so`; do \
(cat $$f | sed -e 's|/usr/lib64/||g' \
-e 's|/usr/lib/||g' \
-e 's|/lib64/||g' \
-e 's|/lib/||g' ) > $$f.tmp ; \
mv $$f.tmp $$f ; \
done)
@mkdir -p $(SYSROOT)/usr/lib
@touch $@
################################################################################
# Create links for ffi header files so that they become visible by default when using the
# devkit.
ifeq ($(ARCH), x86_64)
$(SYSROOT)/usr/include/ffi.h: $(RPMS)
cd $(@D) && rm -f $(@F) && ln -s ../lib/libffi-*/include/$(@F) .
$(SYSROOT)/usr/include/ffitarget.h: $(RPMS)
cd $(@D) && rm -f $(@F) && ln -s ../lib/libffi-*/include/$(@F) .
SYSROOT_LINKS += $(SYSROOT)/usr/include/ffi.h $(SYSROOT)/usr/include/ffitarget.h
endif
################################################################################
# Define marker files for each source package to be compiled
$(foreach dep,$(DEPENDENCIES),$(eval $(dep) = $(TARGETDIR)/$($(dep)_VER).done))
################################################################################
# Default base config
CONFIG = --target=$(TARGET) \
CONFIG = \
--target=$(TARGET) \
--host=$(HOST) --build=$(BUILD) \
--prefix=$(PREFIX)
--prefix=$(PREFIX) \
#
CMAKE_CONFIG = -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$(PREFIX)
@ -357,7 +188,6 @@ BUILDPAR = -j$(NUM_CORES)
MAKECMD =
INSTALLCMD = install
declare_tools = CC$(1)=$(2)gcc LD$(1)=$(2)ld AR$(1)=$(2)ar AS$(1)=$(2)as RANLIB$(1)=$(2)ranlib CXX$(1)=$(2)g++ OBJDUMP$(1)=$(2)objdump
ifeq ($(HOST),$(BUILD))
@ -376,10 +206,10 @@ TOOLS ?= $(call declare_tools,_FOR_TARGET,$(TARGET)-)
# CFLAG_<name> to most likely -m32.
define mk_bfd
$$(info Libs for $(1))
$$(BUILDDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1))/Makefile \
: CFLAGS += $$(CFLAGS_$(1))
$$(BUILDDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1))/Makefile \
: LIBDIRS = --libdir=$(TARGETDIR)/$(1)
$$(BUILDDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1))/Makefile: \
CFLAGS += $$(CFLAGS_$(1))
$$(BUILDDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1))/Makefile: \
LIBDIRS = --libdir=$(TARGETDIR)/$(1)
BFDLIB += $$(TARGETDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1)).done
BFDMAKES += $$(BUILDDIR)/$$(BINUTILS_VER)-$(subst /,-,$(1))/Makefile
@ -389,18 +219,18 @@ endef
$(foreach l,$(LIBDIRS),$(eval $(call mk_bfd,$(l))))
# Only build these two libs.
$(BFDLIB) : MAKECMD = all-libiberty all-bfd
$(BFDLIB) : INSTALLCMD = install-libiberty install-bfd
$(BFDLIB): MAKECMD = all-libiberty all-bfd
$(BFDLIB): INSTALLCMD = install-libiberty install-bfd
# Building targets libbfd + libiberty. HOST==TARGET, i.e not
# for a cross env.
$(BFDMAKES) : CONFIG = --target=$(TARGET) \
$(BFDMAKES): CONFIG = --target=$(TARGET) \
--host=$(TARGET) --build=$(BUILD) \
--prefix=$(TARGETDIR) \
--with-sysroot=$(SYSROOT) \
$(LIBDIRS)
$(BFDMAKES) : TOOLS = $(call declare_tools,_FOR_TARGET,$(TARGET)-) $(call declare_tools,,$(TARGET)-)
$(BFDMAKES): TOOLS = $(call declare_tools,_FOR_TARGET,$(TARGET)-) $(call declare_tools,,$(TARGET)-)
################################################################################
@ -410,50 +240,52 @@ $(GCC) \
$(MPFR) \
$(MPC) \
$(BFDMAKES) \
$(CCACHE) : ENVS += $(TOOLS)
$(CCACHE): ENVS += $(TOOLS)
# libdir to work around hateful bfd stuff installing into wrong dirs...
# ensure we have 64 bit bfd support in the HOST library. I.e our
# compiler on i686 will know 64 bit symbols, BUT later
# we build just the libs again for TARGET, then with whatever the arch
# wants.
$(BUILDDIR)/$(BINUTILS_VER)/Makefile : CONFIG += --enable-64-bit-bfd --libdir=$(PREFIX)/$(word 1,$(LIBDIRS))
$(BUILDDIR)/$(BINUTILS_VER)/Makefile: CONFIG += --enable-64-bit-bfd --libdir=$(PREFIX)/$(word 1,$(LIBDIRS))
ifeq ($(filter $(ARCH), s390x riscv64 ppc64le), )
# gold compiles but cannot link properly on s390x @ gcc 13.2 and Fedore 41
# gold is not available for riscv64 and ppc64le,
# and subsequent linking will fail if we try to enable it.
LINKER_CONFIG := --enable-gold=default
LINKER_CONFIG_ENABLE_GOLD := --enable-gold=default
endif
ifeq ($(filter riscv64 ppc64le s390x armhfp, $(ARCH)), )
ENABLE_MULTILIB := --enable-multilib
endif
# Makefile creation. Simply run configure in build dir.
# Setting CFLAGS to -O2 generates a much faster ld.
$(BFDMAKES) \
$(BUILDDIR)/$(BINUTILS_VER)/Makefile \
: $(BINUTILS_CFG)
$(BUILDDIR)/$(BINUTILS_VER)/Makefile: $(BINUTILS_CFG)
$(info Configuring $@. Log in $(@D)/log.config)
@mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) CFLAGS="-O2 $(CFLAGS)" \
$(BINUTILS_CFG) \
$(CONFIG) \
$(LINKER_CONFIG) \
$(LINKER_CONFIG_ENABLE_GOLD) \
--with-sysroot=$(SYSROOT) \
--disable-nls \
--program-prefix=$(TARGET)- \
--enable-multilib \
$(ENABLE_MULTILIB) \
--enable-threads \
--enable-plugins \
) > $(@D)/log.config 2>&1
@echo 'done'
$(BUILDDIR)/$(MPFR_VER)/Makefile \
: $(MPFR_CFG)
$(BUILDDIR)/$(MPFR_VER)/Makefile: $(MPFR_CFG)
$(info Configuring $@. Log in $(@D)/log.config)
@mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) CFLAGS="$(CFLAGS)" \
$(MPFR_CFG) \
$(CONFIG) \
@ -463,12 +295,11 @@ $(BUILDDIR)/$(MPFR_VER)/Makefile \
) > $(@D)/log.config 2>&1
@echo 'done'
$(BUILDDIR)/$(GMP_VER)/Makefile \
: $(GMP_CFG)
$(BUILDDIR)/$(GMP_VER)/Makefile: $(GMP_CFG)
$(info Configuring $@. Log in $(@D)/log.config)
@mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) CFLAGS="$(CFLAGS)" \
$(GMP_CFG) \
--host=$(HOST) --build=$(BUILD) \
@ -480,12 +311,11 @@ $(BUILDDIR)/$(GMP_VER)/Makefile \
) > $(@D)/log.config 2>&1
@echo 'done'
$(BUILDDIR)/$(MPC_VER)/Makefile \
: $(MPC_CFG)
$(BUILDDIR)/$(MPC_VER)/Makefile: $(MPC_CFG)
$(info Configuring $@. Log in $(@D)/log.config)
@mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) CFLAGS="$(CFLAGS)" \
$(MPC_CFG) \
$(CONFIG) \
@ -499,14 +329,18 @@ $(BUILDDIR)/$(MPC_VER)/Makefile \
# Only valid if glibc target -> linux
# proper destructor handling for c++
ifneq (,$(findstring linux,$(TARGET)))
$(BUILDDIR)/$(GCC_VER)/Makefile : CONFIG += --enable-__cxa_atexit
$(BUILDDIR)/$(GCC_VER)/Makefile: CONFIG += --enable-__cxa_atexit
endif
$(BUILDDIR)/$(GCC_VER)/Makefile: CONFIG += --disable-libgomp
ifeq ($(ARCH), armhfp)
$(BUILDDIR)/$(GCC_VER)/Makefile : CONFIG += --with-float=hard
$(BUILDDIR)/$(GCC_VER)/Makefile: CONFIG += --with-float=hard
endif
ifeq ($(ARCH), riscv64)
$(BUILDDIR)/$(GCC_VER)/Makefile: CONFIG += --disable-libsanitizer
endif
ifneq ($(filter riscv64 ppc64le s390x, $(ARCH)), )
ifneq ($(filter riscv64 ppc64le s390x armhfp, $(ARCH)), )
# We only support 64-bit on these platforms anyway
CONFIG += --disable-multilib
endif
@ -518,12 +352,11 @@ endif
# skip native language.
# and link and assemble with the binutils we created
# earlier, so --with-gnu*
$(BUILDDIR)/$(GCC_VER)/Makefile \
: $(GCC_CFG)
$(BUILDDIR)/$(GCC_VER)/Makefile: $(GCC_CFG)
$(info Configuring $@. Log in $(@D)/log.config)
mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) $(GCC_CFG) $(EXTRA_CFLAGS) \
$(CONFIG) \
--with-sysroot=$(SYSROOT) \
@ -540,12 +373,12 @@ $(BUILDDIR)/$(GCC_VER)/Makefile \
@echo 'done'
# need binutils for gcc
$(GCC) : $(BINUTILS)
$(GCC): $(BINUTILS)
# as of 4.3 or so need these for doing config
$(BUILDDIR)/$(GCC_VER)/Makefile : $(GMP) $(MPFR) $(MPC)
$(MPFR) : $(GMP)
$(MPC) : $(GMP) $(MPFR)
$(BUILDDIR)/$(GCC_VER)/Makefile: $(GMP) $(MPFR) $(MPC)
$(MPFR): $(GMP)
$(MPC): $(GMP) $(MPFR)
################################################################################
# Build gdb but only where host and target match
@ -554,7 +387,7 @@ ifeq ($(HOST), $(TARGET))
$(info Configuring $@. Log in $(@D)/log.config)
mkdir -p $(@D)
( \
cd $(@D) ; \
cd $(@D); \
$(PATHPRE) $(ENVS) CFLAGS="$(CFLAGS)" $(GDB_CFG) \
$(CONFIG) \
--with-sysroot=$(SYSROOT) \
@ -574,14 +407,12 @@ endif
################################################################################
# very straightforward. just build a ccache. it is only for host.
$(BUILDDIR)/$(CCACHE_VER)/Makefile \
: $(CCACHE_SRC_MARKER)
$(BUILDDIR)/$(CCACHE_VER)/Makefile: $(CCACHE_SRC_MARKER)
$(info Configuring $@. Log in $(@D)/log.config)
@mkdir -p $(@D)
@( \
cd $(@D) ; \
$(PATHPRE) $(ENVS) $(CCACHE_CFG) \
$(CCACHE_CONFIG) \
cd $(@D); \
$(PATHPRE) $(ENVS) $(CCACHE_CFG) $(CCACHE_CONFIG) \
) > $(@D)/log.config 2>&1
@echo 'done'
@ -590,13 +421,12 @@ GCC_PATCHED = $(TARGETDIR)/gcc-patched
################################################################################
# For some reason cpp is not created as a target-compiler
ifeq ($(HOST),$(TARGET))
$(GCC_PATCHED) : $(GCC) link_libs
$(GCC_PATCHED): $(LINK_LIBS_MARKER)
@echo -n 'Creating compiler symlinks...'
@for f in cpp; do \
if [ ! -e $(PREFIX)/bin/$(TARGET)-$$f ]; \
then \
if [ ! -e $(PREFIX)/bin/$(TARGET)-$$f ]; then \
cd $(PREFIX)/bin && \
ln -fs $$f $(TARGET)-$$f ; \
ln -fs $$f $(TARGET)-$$f; \
fi \
done
@touch $@
@ -606,19 +436,21 @@ ifeq ($(HOST),$(TARGET))
# Ugly at best. Seems that when we compile host->host compiler, that are NOT
# the BUILD compiler, the result will not try searching for libs in package root.
# "Solve" this by create links from the target libdirs to where they are.
link_libs:
$(LINK_LIBS_MARKER): $(GCC)
@echo -n 'Creating library symlinks...'
@$(foreach l,$(LIBDIRS), \
for f in `cd $(PREFIX)/$(l) && ls`; do \
if [ ! -e $(TARGETDIR)/$(l)/$$f ]; then \
mkdir -p $(TARGETDIR)/$(l) && \
cd $(TARGETDIR)/$(l)/ && \
ln -fs $(if $(findstring /,$(l)),../,)../../$(l)/$$f $$f; \
fi \
done;)
@for l in $(LIBDIRS); do \
for f in `cd $(PREFIX)/$$l && ls`; do \
if [ ! -e $(TARGETDIR)/$$l/$$f ]; then \
mkdir -p $(TARGETDIR)/$$l && \
cd $(TARGETDIR)/$$l/ && \
ln -fs ../../$$l/$$f $$f; \
fi \
done \
done
@touch $@
@echo 'done'
else
$(GCC_PATCHED) :
$(GCC_PATCHED):
@echo 'done'
endif
@ -628,7 +460,7 @@ endif
# make install.
# Use path to our build hosts cross tools
# Always need to build cross tools for build host self.
$(TARGETDIR)/%.done : $(BUILDDIR)/%/Makefile
$(TARGETDIR)/%.done: $(BUILDDIR)/%/Makefile
$(info Building $(basename $@). Log in $(<D)/log.build)
$(PATHPRE) $(ENVS) $(MAKE) $(BUILDPAR) -f $< -C $(<D) $(MAKECMD) $(MAKECMD.$(notdir $@)) > $(<D)/log.build 2>&1
@echo -n 'installing...'
@ -646,26 +478,20 @@ $(PREFIX)/devkit.info:
echo '# This file describes to configure how to interpret the contents of this' >> $@
echo '# devkit' >> $@
echo '' >> $@
echo 'DEVKIT_NAME="$(GCC_VER) - $(LINUX_VERSION)"' >> $@
echo 'DEVKIT_NAME="$(GCC_VER) - $(BASE_OS_DESCRIPTION)"' >> $@
echo 'DEVKIT_TOOLCHAIN_PATH="$$DEVKIT_ROOT/bin"' >> $@
echo 'DEVKIT_SYSROOT="$$DEVKIT_ROOT/$(TARGET)/sysroot"' >> $@
echo 'DEVKIT_EXTRA_PATH="$$DEVKIT_ROOT/bin"' >> $@
################################################################################
# Copy these makefiles into the root of the kit
$(PREFIX)/Makefile: ./Makefile
rm -rf $@
cp $< $@
$(PREFIX)/Tools.gmk: ./Tools.gmk
rm -rf $@
cp $< $@
THESE_MAKEFILES := Makefile $(wildcard *.gmk) patches
COPIED_MAKEFILES := $(addprefix $(PREFIX)/, $(THESE_MAKEFILES))
$(PREFIX)/Tars.gmk: ./Tars.gmk
$(PREFIX)/%: ./%
rm -rf $@
cp $< $@
THESE_MAKEFILES := $(PREFIX)/Makefile $(PREFIX)/Tools.gmk $(PREFIX)/Tars.gmk
cp -r $< $@
################################################################################
@ -682,9 +508,14 @@ ifeq ($(TARGET), $(HOST))
@echo 'Creating missing $* soft link'
ln -s $(TARGET)-$* $@
MISSING_LINKS := $(addprefix $(PREFIX)/bin/, \
MISSING_LINK_NAMES = \
addr2line ar as c++ c++filt dwp elfedit g++ gcc gcc-$(GCC_VER_ONLY) gprof ld ld.bfd \
ld.gold nm objcopy objdump ranlib readelf size strings strip)
nm objcopy objdump ranlib readelf size strings strip
ifneq ($(LINKER_CONFIG_ENABLE_GOLD), )
MISSING_LINK_NAMES += ld.gold
endif
MISSING_LINKS := $(addprefix $(PREFIX)/bin/, $(MISSING_LINK_NAMES))
endif
# Add link to work around "plugin needed to handle lto object" (JDK-8344272)
@ -697,17 +528,14 @@ MISSING_LINKS += $(PREFIX)/lib/bfd-plugins/liblto_plugin.so
################################################################################
bfdlib : $(BFDLIB)
binutils : $(BINUTILS)
rpms : $(RPMS)
libs : $(LIBS)
sysroot : rpms libs
gcc : sysroot $(GCC) $(GCC_PATCHED)
gdb : $(GDB)
all : binutils gcc bfdlib $(PREFIX)/devkit.info $(MISSING_LINKS) $(SYSROOT_LINKS) \
$(THESE_MAKEFILES) gdb
bfdlib: $(BFDLIB)
binutils: $(BINUTILS)
gcc: $(GCC) $(GCC_PATCHED)
gdb: $(GDB)
all: binutils gcc bfdlib gdb \
$(MISSING_LINKS) $(COPIED_MAKEFILES) $(PREFIX)/devkit.info
# this is only built for host. so separate.
ccache : $(CCACHE)
ccache: $(CCACHE)
.PHONY : gcc all binutils bfdlib link_libs rpms libs sysroot
.PHONY: bfdlib binutils gcc gdb all ccache

4
make/modules/jdk.internal.le/Java.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2020, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -25,8 +25,6 @@
################################################################################
DISABLED_WARNINGS_java += dangling-doc-comments this-escape
COPY += .properties .caps .txt
################################################################################

2
make/modules/jdk.jpackage/Java.gmk
View File

@ -25,8 +25,6 @@
################################################################################
DISABLED_WARNINGS_java += dangling-doc-comments
COPY += .gif .png .txt .spec .script .prerm .preinst \
.postrm .postinst .list .sh .desktop .copyright .control .plist .template \
.icns .scpt .wxs .wxl .wxi .wxf .ico .bmp .tiff .service .xsl .js

6
make/test/BuildMicrobenchmark.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2018, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2018, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -83,8 +83,8 @@ $(eval $(call SetupJavaCompilation, BUILD_JDK_MICROBENCHMARK, \
SMALL_JAVA := false, \
CLASSPATH := $(JMH_COMPILE_JARS), \
CREATE_API_DIGEST := true, \
DISABLED_WARNINGS := restricted this-escape processing rawtypes removal cast \
serial preview dangling-doc-comments, \
DISABLED_WARNINGS := restricted this-escape rawtypes removal cast \
serial preview, \
SRC := $(MICROBENCHMARK_SRC), \
BIN := $(MICROBENCHMARK_CLASSES), \
JAVAC_FLAGS := \

3
make/test/BuildTestLib.gmk
View File

@ -1,5 +1,5 @@
#
# Copyright (c) 2015, 2025, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2015, 2026, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
@ -46,7 +46,6 @@ $(eval $(call SetupJavaCompilation, BUILD_WB_JAR, \
SRC := $(TEST_LIB_SOURCE_DIR)/jdk/test/whitebox/, \
BIN := $(TEST_LIB_SUPPORT)/wb_classes, \
JAR := $(TEST_LIB_SUPPORT)/wb.jar, \
DISABLED_WARNINGS := deprecation removal preview, \
JAVAC_FLAGS := --enable-preview, \
))

214
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp
View File

@ -28,7 +28,6 @@
#include "gc/shenandoah/mode/shenandoahMode.hpp"
#include "gc/shenandoah/shenandoahBarrierSet.hpp"
#include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
#include "gc/shenandoah/shenandoahForwarding.hpp"
#include "gc/shenandoah/shenandoahHeap.inline.hpp"
#include "gc/shenandoah/shenandoahHeapRegion.hpp"
#include "gc/shenandoah/shenandoahRuntime.hpp"
@ -174,53 +173,6 @@ void ShenandoahBarrierSetAssembler::satb_barrier(MacroAssembler* masm,
__ bind(done);
}
void ShenandoahBarrierSetAssembler::resolve_forward_pointer(MacroAssembler* masm, Register dst, Register tmp) {
assert(ShenandoahLoadRefBarrier || ShenandoahCASBarrier, "Should be enabled");
Label is_null;
__ cbz(dst, is_null);
resolve_forward_pointer_not_null(masm, dst, tmp);
__ bind(is_null);
}
// IMPORTANT: This must preserve all registers, even rscratch1 and rscratch2, except those explicitly
// passed in.
void ShenandoahBarrierSetAssembler::resolve_forward_pointer_not_null(MacroAssembler* masm, Register dst, Register tmp) {
assert(ShenandoahLoadRefBarrier || ShenandoahCASBarrier, "Should be enabled");
// The below loads the mark word, checks if the lowest two bits are
// set, and if so, clear the lowest two bits and copy the result
// to dst. Otherwise it leaves dst alone.
// Implementing this is surprisingly awkward. I do it here by:
// - Inverting the mark word
// - Test lowest two bits == 0
// - If so, set the lowest two bits
// - Invert the result back, and copy to dst
bool borrow_reg = (tmp == noreg);
if (borrow_reg) {
// No free registers available. Make one useful.
tmp = rscratch1;
if (tmp == dst) {
tmp = rscratch2;
}
__ push(RegSet::of(tmp), sp);
}
assert_different_registers(tmp, dst);
Label done;
__ ldr(tmp, Address(dst, oopDesc::mark_offset_in_bytes()));
__ eon(tmp, tmp, zr);
__ ands(zr, tmp, markWord::lock_mask_in_place);
__ br(Assembler::NE, done);
__ orr(tmp, tmp, markWord::marked_value);
__ eon(dst, tmp, zr);
__ bind(done);
if (borrow_reg) {
__ pop(RegSet::of(tmp), sp);
}
}
void ShenandoahBarrierSetAssembler::load_reference_barrier(MacroAssembler* masm, Register dst, Address load_addr, DecoratorSet decorators) {
assert(ShenandoahLoadRefBarrier, "Should be enabled");
assert(dst != rscratch2, "need rscratch2");
@ -468,166 +420,6 @@ void ShenandoahBarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembl
__ bind(done);
}
// Special Shenandoah CAS implementation that handles false negatives due
// to concurrent evacuation. The service is more complex than a
// traditional CAS operation because the CAS operation is intended to
// succeed if the reference at addr exactly matches expected or if the
// reference at addr holds a pointer to a from-space object that has
// been relocated to the location named by expected. There are two
// races that must be addressed:
// a) A parallel thread may mutate the contents of addr so that it points
// to a different object. In this case, the CAS operation should fail.
// b) A parallel thread may heal the contents of addr, replacing a
// from-space pointer held in addr with the to-space pointer
// representing the new location of the object.
// Upon entry to cmpxchg_oop, it is assured that new_val equals null
// or it refers to an object that is not being evacuated out of
// from-space, or it refers to the to-space version of an object that
// is being evacuated out of from-space.
//
// By default the value held in the result register following execution
// of the generated code sequence is 0 to indicate failure of CAS,
// non-zero to indicate success. If is_cae, the result is the value most
// recently fetched from addr rather than a boolean success indicator.
//
// Clobbers rscratch1, rscratch2
void ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler* masm,
Register addr,
Register expected,
Register new_val,
bool acquire, bool release,
bool is_cae,
Register result) {
Register tmp1 = rscratch1;
Register tmp2 = rscratch2;
bool is_narrow = UseCompressedOops;
Assembler::operand_size size = is_narrow ? Assembler::word : Assembler::xword;
assert_different_registers(addr, expected, tmp1, tmp2);
assert_different_registers(addr, new_val, tmp1, tmp2);
Label step4, done;
// There are two ways to reach this label. Initial entry into the
// cmpxchg_oop code expansion starts at step1 (which is equivalent
// to label step4). Additionally, in the rare case that four steps
// are required to perform the requested operation, the fourth step
// is the same as the first. On a second pass through step 1,
// control may flow through step 2 on its way to failure. It will
// not flow from step 2 to step 3 since we are assured that the
// memory at addr no longer holds a from-space pointer.
//
// The comments that immediately follow the step4 label apply only
// to the case in which control reaches this label by branch from
// step 3.
__ bind (step4);
// Step 4. CAS has failed because the value most recently fetched
// from addr is no longer the from-space pointer held in tmp2. If a
// different thread replaced the in-memory value with its equivalent
// to-space pointer, then CAS may still be able to succeed. The
// value held in the expected register has not changed.
//
// It is extremely rare we reach this point. For this reason, the
// implementation opts for smaller rather than potentially faster
// code. Ultimately, smaller code for this rare case most likely
// delivers higher overall throughput by enabling improved icache
// performance.
// Step 1. Fast-path.
//
// Try to CAS with given arguments. If successful, then we are done.
//
// No label required for step 1.
__ cmpxchg(addr, expected, new_val, size, acquire, release, false, tmp2);
// EQ flag set iff success. tmp2 holds value fetched.
// If expected equals null but tmp2 does not equal null, the
// following branches to done to report failure of CAS. If both
// expected and tmp2 equal null, the following branches to done to
// report success of CAS. There's no need for a special test of
// expected equal to null.
__ br(Assembler::EQ, done);
// if CAS failed, fall through to step 2
// Step 2. CAS has failed because the value held at addr does not
// match expected. This may be a false negative because the value fetched
// from addr (now held in tmp2) may be a from-space pointer to the
// original copy of same object referenced by to-space pointer expected.
//
// To resolve this, it suffices to find the forward pointer associated
// with fetched value. If this matches expected, retry CAS with new
// parameters. If this mismatches, then we have a legitimate
// failure, and we're done.
//
// No need for step2 label.
// overwrite tmp1 with from-space pointer fetched from memory
__ mov(tmp1, tmp2);
if (is_narrow) {
// Decode tmp1 in order to resolve its forward pointer
__ decode_heap_oop(tmp1, tmp1);
}
resolve_forward_pointer(masm, tmp1);
// Encode tmp1 to compare against expected.
__ encode_heap_oop(tmp1, tmp1);
// Does forwarded value of fetched from-space pointer match original
// value of expected? If tmp1 holds null, this comparison will fail
// because we know from step1 that expected is not null. There is
// no need for a separate test for tmp1 (the value originally held
// in memory) equal to null.
__ cmp(tmp1, expected);
// If not, then the failure was legitimate and we're done.
// Branching to done with NE condition denotes failure.
__ br(Assembler::NE, done);
// Fall through to step 3. No need for step3 label.
// Step 3. We've confirmed that the value originally held in memory
// (now held in tmp2) pointed to from-space version of original
// expected value. Try the CAS again with the from-space expected
// value. If it now succeeds, we're good.
//
// Note: tmp2 holds encoded from-space pointer that matches to-space
// object residing at expected. tmp2 is the new "expected".
// Note that macro implementation of __cmpxchg cannot use same register
// tmp2 for result and expected since it overwrites result before it
// compares result with expected.
__ cmpxchg(addr, tmp2, new_val, size, acquire, release, false, noreg);
// EQ flag set iff success. tmp2 holds value fetched, tmp1 (rscratch1) clobbered.
// If fetched value did not equal the new expected, this could
// still be a false negative because some other thread may have
// newly overwritten the memory value with its to-space equivalent.
__ br(Assembler::NE, step4);
if (is_cae) {
// We're falling through to done to indicate success. Success
// with is_cae is denoted by returning the value of expected as
// result.
__ mov(tmp2, expected);
}
__ bind(done);
// At entry to done, the Z (EQ) flag is on iff if the CAS
// operation was successful. Additionally, if is_cae, tmp2 holds
// the value most recently fetched from addr. In this case, success
// is denoted by tmp2 matching expected.
if (is_cae) {
__ mov(result, tmp2);
} else {
__ cset(result, Assembler::EQ);
}
}
void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
Register start, Register count, Register scratch) {
assert(ShenandoahCardBarrier, "Should have been checked by caller");
@ -868,7 +660,7 @@ void ShenandoahBarrierSetAssembler::load_c2(const MachNode* node, MacroAssembler
void ShenandoahBarrierSetAssembler::store_c2(const MachNode* node, MacroAssembler* masm, Address dst, bool dst_narrow,
Register src, bool src_narrow, Register tmp1, Register tmp2, Register tmp3, bool is_volatile) {
ShenandoahBarrierStubC2::store_pre(masm, node, tmp1, dst, tmp2, tmp3, dst_narrow);
ShenandoahBarrierStubC2::store_pre(masm, node, dst, tmp1, tmp2, tmp3, dst_narrow);
// Do the actual store
if (dst_narrow) {
@ -906,7 +698,7 @@ void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, Mac
Register oldval, Register newval, Register tmp1, Register tmp2, Register tmp3, bool exchange, bool narrow, bool weak, bool acquire) {
Assembler::operand_size op_size = narrow ? Assembler::word : Assembler::xword;
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, addr, tmp2, tmp3, narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, tmp1, tmp2, tmp3, narrow);
// CAS!
__ cmpxchg(addr, oldval, newval, op_size, acquire, /* release */ true, weak, exchange ? res : noreg);
@ -924,7 +716,7 @@ void ShenandoahBarrierSetAssembler::get_and_set_c2(const MachNode* node, MacroAs
Register newval, Register addr, Register tmp1, Register tmp2, Register tmp3, bool is_acquire) {
bool is_narrow = node->bottom_type()->isa_narrowoop();
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, addr, tmp2, tmp3, is_narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, tmp1, tmp2, tmp3, is_narrow);
if (is_narrow) {
if (is_acquire) {

4
src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.hpp
View File

@ -53,8 +53,6 @@ private:
void card_barrier(MacroAssembler* masm, Register obj);
void resolve_forward_pointer(MacroAssembler* masm, Register dst, Register tmp = noreg);
void resolve_forward_pointer_not_null(MacroAssembler* masm, Register dst, Register tmp = noreg);
void load_reference_barrier(MacroAssembler* masm, Register dst, Address load_addr, DecoratorSet decorators);
void gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
@ -76,8 +74,6 @@ public:
Register obj, Register tmp, Label& slowpath);
virtual void try_peek_weak_handle_in_nmethod(MacroAssembler* masm, Register weak_handle, Register obj,
Register tmp, Label& slow_path);
void cmpxchg_oop(MacroAssembler* masm, Register addr, Register expected, Register new_val,
bool acquire, bool release, bool is_cae, Register result);
#ifdef COMPILER1
void gen_pre_barrier_stub(LIR_Assembler* ce, ShenandoahPreBarrierStub* stub);

71
src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
View File

@ -2147,7 +2147,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
Register offset = rscratch2;
Label L_loop_search_receiver, L_loop_search_empty;
Label L_restart, L_found_recv, L_found_empty, L_polymorphic, L_count_update;
Label L_restart, L_found_recv, L_found_empty, L_count_update;
// The code here recognizes three major cases:
// A. Fastest: receiver found in the table
@ -2177,21 +2177,20 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// if (receiver(i) == recv) goto found_recv(i);
// }
//
// // Fast: no receiver, but profile is full
// // Fast: no receiver, but profile is not full
// for (i = 0; i < receiver_count(); i++) {
// if (receiver(i) == null) goto found_null(i);
// }
// goto polymorphic
//
// // Slow: profile is full, polymorphic case
// count++;
// return
//
// // Slow: try to install receiver
// found_null(i):
// CAS(&receiver(i), null, recv);
// goto restart
//
// polymorphic:
// count++;
// return
//
// found_recv(i):
// *receiver_count(i)++
//
@ -2208,7 +2207,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
sub(rscratch1, offset, end_receiver_offset);
cbnz(rscratch1, L_loop_search_receiver);
// Fast: no receiver, but profile is full
// Fast: no receiver, but profile is not full
mov(offset, base_receiver_offset);
bind(L_loop_search_empty);
ldr(rscratch1, Address(mdp, offset));
@ -2216,9 +2215,13 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
add(offset, offset, receiver_step);
sub(rscratch1, offset, end_receiver_offset);
cbnz(rscratch1, L_loop_search_empty);
b(L_polymorphic);
// Slow: try to install receiver
// Slow: Receiver is not found and table is full.
// Increment polymorphic counter instead of receiver slot.
mov(offset, poly_count_offset);
b(L_count_update);
// Slowest: try to install receiver
bind(L_found_empty);
// Atomically swing receiver slot: null -> recv.
@ -2237,17 +2240,11 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// and just restart the search from the beginning.
b(L_restart);
// Counter updates:
// Increment polymorphic counter instead of receiver slot.
bind(L_polymorphic);
mov(offset, poly_count_offset);
b(L_count_update);
// Found a receiver, convert its slot offset to corresponding count offset.
bind(L_found_recv);
add(offset, offset, receiver_to_count_step);
// Finally, update the counter
bind(L_count_update);
increment(Address(mdp, offset), DataLayout::counter_increment);
}
@ -2816,7 +2813,7 @@ void MacroAssembler::decrementw(Register reg, int value)
{
if (value < 0) { incrementw(reg, -value); return; }
if (value == 0) { return; }
if (value < (1 << 12)) { subw(reg, reg, value); return; }
if (value < (1 << 24)) { subw(reg, reg, value); return; }
/* else */ {
guarantee(reg != rscratch2, "invalid dst for register decrement");
movw(rscratch2, (unsigned)value);
@ -2828,7 +2825,7 @@ void MacroAssembler::decrement(Register reg, int value)
{
if (value < 0) { increment(reg, -value); return; }
if (value == 0) { return; }
if (value < (1 << 12)) { sub(reg, reg, value); return; }
if (value < (1 << 24)) { sub(reg, reg, value); return; }
/* else */ {
assert(reg != rscratch2, "invalid dst for register decrement");
mov(rscratch2, (uint64_t)value);
@ -2840,7 +2837,7 @@ void MacroAssembler::decrementw(Address dst, int value)
{
assert(!dst.uses(rscratch1), "invalid dst for address decrement");
if (dst.getMode() == Address::literal) {
assert(abs(value) < (1 << 12), "invalid value and address mode combination");
assert(abs(value) < (1 << 24), "invalid value and address mode combination");
lea(rscratch2, dst);
dst = Address(rscratch2);
}
@ -2853,7 +2850,7 @@ void MacroAssembler::decrement(Address dst, int value)
{
assert(!dst.uses(rscratch1), "invalid address for decrement");
if (dst.getMode() == Address::literal) {
assert(abs(value) < (1 << 12), "invalid value and address mode combination");
assert(abs(value) < (1 << 24), "invalid value and address mode combination");
lea(rscratch2, dst);
dst = Address(rscratch2);
}
@ -2866,7 +2863,7 @@ void MacroAssembler::incrementw(Register reg, int value)
{
if (value < 0) { decrementw(reg, -value); return; }
if (value == 0) { return; }
if (value < (1 << 12)) { addw(reg, reg, value); return; }
if (value < (1 << 24)) { addw(reg, reg, value); return; }
/* else */ {
assert(reg != rscratch2, "invalid dst for register increment");
movw(rscratch2, (unsigned)value);
@ -2878,7 +2875,7 @@ void MacroAssembler::increment(Register reg, int value)
{
if (value < 0) { decrement(reg, -value); return; }
if (value == 0) { return; }
if (value < (1 << 12)) { add(reg, reg, value); return; }
if (value < (1 << 24)) { add(reg, reg, value); return; }
/* else */ {
assert(reg != rscratch2, "invalid dst for register increment");
movw(rscratch2, (unsigned)value);
@ -2886,30 +2883,34 @@ void MacroAssembler::increment(Register reg, int value)
}
}
void MacroAssembler::incrementw(Address dst, int value)
void MacroAssembler::incrementw(Address dst, int value, Register result)
{
assert(!dst.uses(rscratch1), "invalid dst for address increment");
assert(!dst.uses(result), "invalid dst for address increment");
assert(result->is_valid(), "must be");
assert_different_registers(result, rscratch2);
if (dst.getMode() == Address::literal) {
assert(abs(value) < (1 << 12), "invalid value and address mode combination");
assert(abs(value) < (1 << 24), "invalid value and address mode combination");
lea(rscratch2, dst);
dst = Address(rscratch2);
}
ldrw(rscratch1, dst);
incrementw(rscratch1, value);
strw(rscratch1, dst);
ldrw(result, dst);
incrementw(result, value);
strw(result, dst);
}
void MacroAssembler::increment(Address dst, int value)
void MacroAssembler::increment(Address dst, int value, Register result)
{
assert(!dst.uses(rscratch1), "invalid dst for address increment");
assert(!dst.uses(result), "invalid dst for address increment");
assert(result->is_valid(), "must be");
assert_different_registers(result, rscratch2);
if (dst.getMode() == Address::literal) {
assert(abs(value) < (1 << 12), "invalid value and address mode combination");
assert(abs(value) < (1 << 24), "invalid value and address mode combination");
lea(rscratch2, dst);
dst = Address(rscratch2);
}
ldr(rscratch1, dst);
increment(rscratch1, value);
str(rscratch1, dst);
ldr(result, dst);
increment(result, value);
str(result, dst);
}
// Push lots of registers in the bit set supplied. Don't push sp.

25
src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
View File

@ -482,6 +482,25 @@ class MacroAssembler: public Assembler {
WRAP(smaddl) WRAP(smsubl) WRAP(umaddl) WRAP(umsubl)
#undef WRAP
using Assembler::andw, Assembler::andr;
void andw(Register Rd, Register Rn, uint64_t imm) {
if (operand_valid_for_logical_immediate(/*is32*/true, imm)) {
Assembler::andw(Rd, Rn, imm);
} else {
assert(Rd != Rn, "must be");
movw(Rd, imm);
andw(Rd, Rn, Rd);
}
}
void andr(Register Rd, Register Rn, uint64_t imm) {
if (operand_valid_for_logical_immediate(/*is32*/false, imm)) {
Assembler::andr(Rd, Rn, imm);
} else {
assert(Rd != Rn, "must be");
mov(Rd, imm);
andr(Rd, Rn, Rd);
}
}
// macro assembly operations needed for aarch64
@ -743,7 +762,7 @@ public:
// n.b. increment/decrement calls with an Address destination will
// need to use a scratch register to load the value to be
// incremented. increment/decrement calls which add or subtract a
// constant value greater than 2^12 will need to use a 2nd scratch
// constant value greater than 2^24 will need to use a 2nd scratch
// register to hold the constant. so, a register increment/decrement
// may trash rscratch2 and an address increment/decrement trash
// rscratch and rscratch2
@ -754,11 +773,11 @@ public:
void decrement(Register reg, int value = 1);
void decrement(Address dst, int value = 1);
void incrementw(Address dst, int value = 1);
void incrementw(Address dst, int value = 1, Register result = rscratch1);
void incrementw(Register reg, int value = 1);
void increment(Register reg, int value = 1);
void increment(Address dst, int value = 1);
void increment(Address dst, int value = 1, Register result = rscratch1);
// Alignment

185
src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.cpp
View File

@ -31,7 +31,6 @@
#include "gc/shenandoah/mode/shenandoahMode.hpp"
#include "gc/shenandoah/shenandoahBarrierSet.hpp"
#include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
#include "gc/shenandoah/shenandoahForwarding.hpp"
#include "gc/shenandoah/shenandoahHeap.hpp"
#include "gc/shenandoah/shenandoahHeap.inline.hpp"
#include "gc/shenandoah/shenandoahHeapRegion.hpp"
@ -335,43 +334,6 @@ void ShenandoahBarrierSetAssembler::satb_barrier_impl(MacroAssembler *masm, Deco
__ bind(skip_barrier);
}
void ShenandoahBarrierSetAssembler::resolve_forward_pointer_not_null(MacroAssembler *masm, Register dst, Register tmp) {
__ block_comment("resolve_forward_pointer_not_null (shenandoahgc) {");
Register tmp1 = tmp,
R0_tmp2 = R0;
assert_different_registers(dst, tmp1, R0_tmp2, noreg);
// If the object has been evacuated, the mark word layout is as follows:
// | forwarding pointer (62-bit) | '11' (2-bit) |
// The invariant that stack/thread pointers have the lowest two bits cleared permits retrieving
// the forwarding pointer solely by inversing the lowest two bits.
// This invariant follows inevitably from hotspot's minimal alignment.
assert(markWord::marked_value <= (unsigned long) MinObjAlignmentInBytes,
"marked value must not be higher than hotspot's minimal alignment");
Label done;
// Load the object's mark word.
__ ld(tmp1, oopDesc::mark_offset_in_bytes(), dst);
// Load the bit mask for the lock bits.
__ li(R0_tmp2, markWord::lock_mask_in_place);
// Check whether all bits matching the bit mask are set.
// If that is the case, the object has been evacuated and the most significant bits form the forward pointer.
__ andc_(R0_tmp2, R0_tmp2, tmp1);
assert(markWord::lock_mask_in_place == markWord::marked_value,
"marked value must equal the value obtained when all lock bits are being set");
__ xori(tmp1, tmp1, markWord::lock_mask_in_place);
__ isel(dst, CR0, Assembler::equal, false, tmp1);
__ bind(done);
__ block_comment("} resolve_forward_pointer_not_null (shenandoahgc)");
}
// base: Base register of the reference's address.
// ind_or_offs: Index or offset of the reference's address (load mode).
// dst: Reference's address. In case the object has been evacuated, this is the to-space version
@ -693,125 +655,6 @@ void ShenandoahBarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembl
__ block_comment("} try_peek_weak_handle_in_nmethod (shenandoahgc)");
}
// Special shenandoah CAS implementation that handles false negatives due
// to concurrent evacuation. That is, the CAS operation is intended to succeed in
// the following scenarios (success criteria):
// s1) The reference pointer ('base_addr') equals the expected ('expected') pointer.
// s2) The reference pointer refers to the from-space version of an already-evacuated
// object, whereas the expected pointer refers to the to-space version of the same object.
// Situations in which the reference pointer refers to the to-space version of an object
// and the expected pointer refers to the from-space version of the same object can not occur due to
// shenandoah's strong to-space invariant. This also implies that the reference stored in 'new_val'
// can not refer to the from-space version of an already-evacuated object.
//
// To guarantee correct behavior in concurrent environments, two races must be addressed:
// r1) A concurrent thread may heal the reference pointer (i.e., it is no longer referring to the
// from-space version but to the to-space version of the object in question).
// In this case, the CAS operation should succeed.
// r2) A concurrent thread may mutate the reference (i.e., the reference pointer refers to an entirely different object).
// In this case, the CAS operation should fail.
//
// By default, the value held in the 'result' register is zero to indicate failure of CAS,
// non-zero to indicate success. If 'is_cae' is set, the result is the most recently fetched
// value from 'base_addr' rather than a boolean success indicator.
void ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler *masm, Register base_addr,
Register expected, Register new_val, Register tmp1, Register tmp2,
bool is_cae, Register result) {
__ block_comment("cmpxchg_oop (shenandoahgc) {");
assert_different_registers(base_addr, new_val, tmp1, tmp2, result, R0);
assert_different_registers(base_addr, expected, tmp1, tmp2, result, R0);
// Potential clash of 'success_flag' and 'tmp' is being accounted for.
Register success_flag = is_cae ? noreg : result,
current_value = is_cae ? result : tmp1,
tmp = is_cae ? tmp1 : result,
initial_value = tmp2;
Label done, step_four;
__ bind(step_four);
/* ==== Step 1 ("Standard" CAS) ==== */
// Fast path: The values stored in 'expected' and 'base_addr' are equal.
// Given that 'expected' must refer to the to-space object of an evacuated object (strong to-space invariant),
// no special processing is required.
if (UseCompressedOops) {
__ cmpxchgw(CR0, current_value, expected, new_val, base_addr, MacroAssembler::MemBarNone,
false, success_flag, nullptr, true);
} else {
__ cmpxchgd(CR0, current_value, expected, new_val, base_addr, MacroAssembler::MemBarNone,
false, success_flag, nullptr, true);
}
// Skip the rest of the barrier if the CAS operation succeeds immediately.
// If it does not, the value stored at the address is either the from-space pointer of the
// referenced object (success criteria s2)) or simply another object.
__ beq(CR0, done);
/* ==== Step 2 (Null check) ==== */
// The success criteria s2) cannot be matched with a null pointer
// (null pointers cannot be subject to concurrent evacuation). The failure of the CAS operation is thus legitimate.
__ cmpdi(CR0, current_value, 0);
__ beq(CR0, done);
/* ==== Step 3 (reference pointer refers to from-space version; success criteria s2)) ==== */
// To check whether the reference pointer refers to the from-space version, the forward
// pointer of the object referred to by the reference is resolved and compared against the expected pointer.
// If this check succeed, another CAS operation is issued with the from-space pointer being the expected pointer.
//
// Save the potential from-space pointer.
__ mr(initial_value, current_value);
// Resolve forward pointer.
if (UseCompressedOops) { __ decode_heap_oop_not_null(current_value); }
resolve_forward_pointer_not_null(masm, current_value, tmp);
if (UseCompressedOops) { __ encode_heap_oop_not_null(current_value); }
if (!is_cae) {
// 'success_flag' was overwritten by call to 'resovle_forward_pointer_not_null'.
// Load zero into register for the potential failure case.
__ li(success_flag, 0);
}
__ cmpd(CR0, current_value, expected);
__ bne(CR0, done);
// Discard fetched value as it might be a reference to the from-space version of an object.
if (UseCompressedOops) {
__ cmpxchgw(CR0, R0, initial_value, new_val, base_addr, MacroAssembler::MemBarNone,
false, success_flag);
} else {
__ cmpxchgd(CR0, R0, initial_value, new_val, base_addr, MacroAssembler::MemBarNone,
false, success_flag);
}
/* ==== Step 4 (Retry CAS with to-space pointer (success criteria s2) under race r1)) ==== */
// The reference pointer could have been healed whilst the previous CAS operation was being performed.
// Another CAS operation must thus be issued with the to-space pointer being the expected pointer.
// If that CAS operation fails as well, race r2) must have occurred, indicating that
// the operation failure is legitimate.
//
// To keep the code's size small and thus improving cache (icache) performance, this highly
// unlikely case should be handled by the smallest possible code. Instead of emitting a third,
// explicit CAS operation, the code jumps back and reuses the first CAS operation (step 1)
// (passed arguments are identical).
//
// A failure of the CAS operation in step 1 would imply that the overall CAS operation is supposed
// to fail. Jumping back to step 1 requires, however, that step 2 and step 3 are re-executed as well.
// It is thus important to ensure that a re-execution of those steps does not put program correctness
// at risk:
// - Step 2: Either terminates in failure (desired result) or falls through to step 3.
// - Step 3: Terminates if the comparison between the forwarded, fetched pointer and the expected value
// fails. Unless the reference has been updated in the meanwhile once again, this is
// guaranteed to be the case.
// In case of a concurrent update, the CAS would be retried again. This is legitimate
// in terms of program correctness (even though it is not desired).
__ bne(CR0, step_four);
__ bind(done);
__ block_comment("} cmpxchg_oop (shenandoahgc)");
}
void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
Register addr, Register count, Register preserve) {
assert(ShenandoahCardBarrier, "Should have been checked by caller");
@ -1115,7 +958,7 @@ void ShenandoahBarrierSetAssembler::load_c2(const MachNode* node, MacroAssembler
void ShenandoahBarrierSetAssembler::store_c2(const MachNode* node, MacroAssembler* masm,
Register dst, int disp, bool dst_narrow, Register src, bool src_narrow, Register tmp1, Register tmp2, Register tmp3) {
ShenandoahBarrierStubC2::store_pre(masm, node, tmp1, Address(dst, disp), tmp2, tmp3, dst_narrow);
ShenandoahBarrierStubC2::store_pre(masm, node, Address(dst, disp), tmp1, tmp2, tmp3, dst_narrow);
if (dst_narrow && !src_narrow) {
// Need to encode into tmp, because we cannot clobber src.
@ -1134,39 +977,41 @@ void ShenandoahBarrierSetAssembler::store_c2(const MachNode* node, MacroAssemble
ShenandoahBarrierStubC2::store_post(masm, node, Address(dst, disp), tmp1, tmp2);
}
void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, MacroAssembler* masm, Register res, Register addr, Register oldval,
Register newval, Register tmp1, Register tmp2, Register tmp3, bool exchange, bool narrow, bool weak, bool acquire) {
void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, MacroAssembler* masm, Register res, Register addr,
Register oldval, Register newval, Register tmp1, Register tmp2, bool exchange, bool narrow, bool weak, bool acquire) {
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, addr, tmp2, tmp3, narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, res, tmp1, tmp2, narrow);
Register dest_current = exchange ? res : R0;
Register int_flag = exchange ? noreg : res;
int semantics = MacroAssembler::MemBarNone;
Register dest_current = exchange ? res : R0;
Label no_update;
int semantics = MacroAssembler::MemBarNone;
if (acquire) {
semantics = support_IRIW_for_not_multiple_copy_atomic_cpu ?
MacroAssembler::MemBarAcq : MacroAssembler::MemBarFenceAfter;
}
if (!exchange) { __ li(res, 0); }
if (narrow) {
// CmpxchgX sets CR0 to cmpX(src1, src2) and Rres to 'true'/'false'.
__ cmpxchgw(CR0, dest_current, oldval, newval, addr,
semantics, MacroAssembler::cmpxchgx_hint_atomic_update(),
int_flag, nullptr, true, weak);
noreg, &no_update, true, weak);
} else {
// CmpxchgX sets CR0 to cmpX(src1, src2) and Rres to 'true'/'false'.
__ cmpxchgd(CR0, dest_current, oldval, newval, addr,
semantics, MacroAssembler::cmpxchgx_hint_atomic_update(),
int_flag, nullptr, true, weak);
noreg, &no_update, true, weak);
}
if (!exchange) { __ li(res, 1); }
ShenandoahBarrierStubC2::load_store_post(masm, node, Address(addr, 0), tmp1, tmp2);
__ bind(no_update);
}
void ShenandoahBarrierSetAssembler::get_and_set_c2(const MachNode* node, MacroAssembler* masm, Register preval, Register newval, Register addr, Register tmp1, Register tmp2, Register tmp3) {
void ShenandoahBarrierSetAssembler::get_and_set_c2(const MachNode* node, MacroAssembler* masm, Register preval, Register newval, Register addr, Register tmp1, Register tmp2) {
bool is_narrow = node->bottom_type()->isa_narrowoop();
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, addr, tmp2, tmp3, is_narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, preval, tmp1, tmp2, is_narrow);
if (is_narrow) {
__ getandsetw(preval, newval, addr, MacroAssembler::cmpxchgx_hint_atomic_update());

11
src/hotspot/cpu/ppc/gc/shenandoah/shenandoahBarrierSetAssembler_ppc.hpp
View File

@ -69,8 +69,6 @@ private:
MacroAssembler::PreservationLevel preservation_level);
/* ==== Helper methods for barrier implementations ==== */
void resolve_forward_pointer_not_null(MacroAssembler* masm, Register dst, Register tmp);
void gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
Register addr, Register count,
Register preserve);
@ -103,11 +101,6 @@ public:
Register tmp1, Register tmp2,
MacroAssembler::PreservationLevel preservation_level);
/* ==== Helper methods used by C1 and C2 ==== */
void cmpxchg_oop(MacroAssembler* masm, Register base_addr, Register expected, Register new_val,
Register tmp1, Register tmp2,
bool is_cae, Register result);
/* ==== Access api ==== */
virtual void arraycopy_prologue(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
Register src, Register dst, Register count,
@ -140,10 +133,10 @@ public:
Register dst, int disp, bool dst_narrow, Register src, bool src_narrow, Register tmp1, Register tmp2, Register tmp3);
void compare_and_set_c2(const MachNode* node, MacroAssembler* masm, Register res, Register addr, Register oldval,
Register newval, Register tmp1, Register tmp2, Register tmp3, bool exchange, bool narrow, bool weak, bool acquire);
Register newval, Register tmp1, Register tmp2, bool exchange, bool narrow, bool weak, bool acquire);
void get_and_set_c2(const MachNode* node, MacroAssembler* masm,
Register preval, Register newval, Register addr, Register tmp1, Register tmp2, Register tmp3);
Register preval, Register newval, Register addr, Register tmp1, Register tmp2);
#endif // COMPILER2
};

149
src/hotspot/cpu/ppc/gc/shenandoah/shenandoah_ppc.ad
View File

@ -1,6 +1,6 @@
//
// Copyright (c) 2018, 2021, Red Hat, Inc. All rights reserved.
// Copyright (c) 2012, 2021 SAP SE. All rights reserved.
// Copyright (c) 2012, 2026 SAP SE. All rights reserved.
// DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
//
// This code is free software; you can redistribute it and/or modify it
@ -201,10 +201,12 @@ instruct encodePAndStoreN_shenandoah(memory dst, iRegPsrc src, iRegPdstNoScratch
// ---------------------- LOAD-STORES -----------------------------------
//
instruct compareAndSwapN_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
// Strong CAS also handles WeakCompareAndSwap* on PPC, see JDK-8385633.
instruct compareAndSwapN_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndSwapN mem_ptr (Binary src1 src2)));
match(Set res (WeakCompareAndSwapN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0); // TEMP_DEF to avoid jump
format %{ "CMPXCHGW $res, $mem_ptr, $src1, $src2; as bool" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -214,7 +216,6 @@ instruct compareAndSwapN_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_pt
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ true,
/* weak */ false,
@ -223,10 +224,11 @@ instruct compareAndSwapN_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_pt
ins_pipe(pipe_class_default);
%}
instruct compareAndSwapN_acq_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndSwapN_acq_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndSwapN mem_ptr (Binary src1 src2)));
match(Set res (WeakCompareAndSwapN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0); // TEMP_DEF to avoid jump
format %{ "CMPXCHGW $res, $mem_ptr, $src1, $src2; as bool" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -236,7 +238,6 @@ instruct compareAndSwapN_acq_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst me
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ true,
/* weak */ false,
@ -245,9 +246,10 @@ instruct compareAndSwapN_acq_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst me
ins_pipe(pipe_class_default);
%}
instruct compareAndSwapP_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndSwapP_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndSwapP mem_ptr (Binary src1 src2)));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
match(Set res (WeakCompareAndSwapP mem_ptr (Binary src1 src2)));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0); // TEMP_DEF to avoid jump
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
format %{ "CMPXCHGD $res, $mem_ptr, $src1, $src2; as bool; ptr" %}
ins_encode %{
@ -258,7 +260,6 @@ instruct compareAndSwapP_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_pt
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ false,
/* weak */ false,
@ -267,9 +268,10 @@ instruct compareAndSwapP_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_pt
ins_pipe(pipe_class_default);
%}
instruct compareAndSwapP_acq_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndSwapP_acq_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndSwapP mem_ptr (Binary src1 src2)));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
match(Set res (WeakCompareAndSwapP mem_ptr (Binary src1 src2)));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0); // TEMP_DEF to avoid jump
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
format %{ "CMPXCHGD $res, $mem_ptr, $src1, $src2; as bool; ptr" %}
ins_encode %{
@ -280,7 +282,6 @@ instruct compareAndSwapP_acq_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst me
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ false,
/* weak */ false,
@ -289,98 +290,10 @@ instruct compareAndSwapP_acq_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst me
ins_pipe(pipe_class_default);
%}
instruct weakCompareAndSwapN_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
match(Set res (WeakCompareAndSwapN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
format %{ "weak CMPXCHGW acq $res, $mem_ptr, $src1, $src2; as bool" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
$res$$Register,
$mem_ptr$$Register,
$src1$$Register,
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ true,
/* weak */ true,
/* acquire */ false);
%}
ins_pipe(pipe_class_default);
%}
instruct weakCompareAndSwapN_acq_regP_regN_regN_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
match(Set res (WeakCompareAndSwapN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
format %{ "weak CMPXCHGW acq $res, $mem_ptr, $src1, $src2; as bool" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
$res$$Register,
$mem_ptr$$Register,
$src1$$Register,
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ true,
/* weak */ true,
/* acquire */ true);
%}
ins_pipe(pipe_class_default);
%}
instruct weakCompareAndSwapP_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
match(Set res (WeakCompareAndSwapP mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
format %{ "weak CMPXCHGD $res, $mem_ptr, $src1, $src2; as bool; ptr" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
$res$$Register,
$mem_ptr$$Register,
$src1$$Register,
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ false,
/* weak */ true,
/* acquire */ false);
%}
ins_pipe(pipe_class_default);
%}
instruct weakCompareAndSwapP_acq_regP_regP_regP_shenandoah(iRegIdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
match(Set res (WeakCompareAndSwapP mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0); // TEMP_DEF to avoid jump
format %{ "weak CMPXCHGD $res, $mem_ptr, $src1, $src2; as bool; ptr" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
$res$$Register,
$mem_ptr$$Register,
$src1$$Register,
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ false,
/* is_narrow */ false,
/* weak */ true,
/* acquire */ true);
%}
ins_pipe(pipe_class_default);
%}
instruct compareAndExchangeN_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndExchangeN_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndExchangeN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "CMPXCHGW $res, $mem_ptr, $src1, $src2; as narrow oop" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -390,7 +303,6 @@ instruct compareAndExchangeN_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst me
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ true,
/* is_narrow */ true,
/* weak */ false,
@ -399,10 +311,10 @@ instruct compareAndExchangeN_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst me
ins_pipe(pipe_class_default);
%}
instruct compareAndExchangeN_acq_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndExchangeN_acq_regP_regN_regN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src1, iRegNsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndExchangeN mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "CMPXCHGW $res, $mem_ptr, $src1, $src2; as narrow oop" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -412,7 +324,6 @@ instruct compareAndExchangeN_acq_regP_regN_regN_shenandoah(iRegNdst res, iRegPds
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ true,
/* is_narrow */ true,
/* weak */ false,
@ -421,10 +332,10 @@ instruct compareAndExchangeN_acq_regP_regN_regN_shenandoah(iRegNdst res, iRegPds
ins_pipe(pipe_class_default);
%}
instruct compareAndExchangeP_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndExchangeP_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndExchangeP mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && !need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "CMPXCHGD $res, $mem_ptr, $src1, $src2; as ptr; ptr" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -434,7 +345,6 @@ instruct compareAndExchangeP_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst me
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ true,
/* is_narrow */ false,
/* weak */ false,
@ -443,10 +353,10 @@ instruct compareAndExchangeP_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst me
ins_pipe(pipe_class_default);
%}
instruct compareAndExchangeP_acq_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct compareAndExchangeP_acq_regP_regP_regP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src1, iRegPsrc src2, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (CompareAndExchangeP mem_ptr (Binary src1 src2)));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0) && need_acquire_load_store(n));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "CMPXCHGD $res, $mem_ptr, $src1, $src2; as ptr; ptr" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->compare_and_set_c2(this, masm,
@ -456,7 +366,6 @@ instruct compareAndExchangeP_acq_regP_regP_regP_shenandoah(iRegPdst res, iRegPds
$src2$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register,
/* exchange */ true,
/* is_narrow */ false,
/* weak */ false,
@ -465,10 +374,10 @@ instruct compareAndExchangeP_acq_regP_regP_regP_shenandoah(iRegPdst res, iRegPds
ins_pipe(pipe_class_default);
%}
instruct getAndSetP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct getAndSetP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (GetAndSetP mem_ptr src));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "GetAndSetP $res, $mem_ptr, $src" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->get_and_set_c2(this, masm,
@ -476,16 +385,15 @@ instruct getAndSetP_shenandoah(iRegPdst res, iRegPdst mem_ptr, iRegPsrc src, iRe
$src$$Register,
$mem_ptr$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register);
$tmp2$$Register);
%}
ins_pipe(pipe_class_default);
%}
instruct getAndSetN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, iRegPdstNoScratch tmp3, flagsRegCR0 cr0) %{
instruct getAndSetN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src, iRegPdstNoScratch tmp1, iRegPdstNoScratch tmp2, flagsRegCR0 cr0) %{
match(Set res (GetAndSetN mem_ptr src));
predicate(UseShenandoahGC && (n->as_LoadStore()->barrier_data() != 0));
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, TEMP tmp3, KILL cr0);
effect(TEMP_DEF res, TEMP tmp1, TEMP tmp2, KILL cr0);
format %{ "GetAndSetN $res, $mem_ptr, $src" %}
ins_encode %{
ShenandoahBarrierSet::assembler()->get_and_set_c2(this, masm,
@ -493,8 +401,7 @@ instruct getAndSetN_shenandoah(iRegNdst res, iRegPdst mem_ptr, iRegNsrc src, iRe
$src$$Register,
$mem_ptr$$Register,
$tmp1$$Register,
$tmp2$$Register,
$tmp3$$Register);
$tmp2$$Register);
%}
ins_pipe(pipe_class_default);
%}

18
src/hotspot/cpu/ppc/macroAssembler_ppc.cpp
View File

@ -4411,21 +4411,20 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// if (receiver(i) == recv) goto found_recv(i);
// }
//
// // Fast: no receiver, but profile is full
// // Fast: no receiver, but profile is not full
// for (i = 0; i < receiver_count(); i++) {
// if (receiver(i) == null) goto found_null(i);
// }
// goto polymorphic
//
// // Slow: profile is full, polymorphic case
// count++;
// return
//
// // Slow: try to install receiver
// found_null(i):
// CAS(&receiver(i), null, recv);
// goto restart
//
// polymorphic:
// count++;
// return
//
// found_recv(i):
// *receiver_count(i)++
//
@ -4466,11 +4465,12 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
addi(offset, offset, receiver_step);
bdnz(L_loop_search_empty);
// Polymorphic: Increment polymorphic counter instead of receiver slot.
// Slow: Receiver is not found and table is full.
// Increment polymorphic counter instead of receiver slot.
li(offset, poly_count_offset);
b(L_count_update);
// Slow: try to install receiver
// Slowest: try to install receiver
bind(L_found_empty);
// Atomically swing receiver slot: null -> recv.
@ -4491,7 +4491,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
bind(L_found_recv);
addi(offset, offset, receiver_to_count_step);
// Counter update
// Finally, update the counter
bind(L_count_update);
increment_mem64(mdp, offset, DataLayout::counter_increment, /* temp */ (count != noreg) ? count : recv);
}

138
src/hotspot/cpu/riscv/gc/shenandoah/shenandoahBarrierSetAssembler_riscv.cpp
View File

@ -28,7 +28,6 @@
#include "gc/shenandoah/mode/shenandoahMode.hpp"
#include "gc/shenandoah/shenandoahBarrierSet.hpp"
#include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
#include "gc/shenandoah/shenandoahForwarding.hpp"
#include "gc/shenandoah/shenandoahHeap.inline.hpp"
#include "gc/shenandoah/shenandoahHeapRegion.hpp"
#include "gc/shenandoah/shenandoahRuntime.hpp"
@ -174,53 +173,6 @@ void ShenandoahBarrierSetAssembler::satb_barrier(MacroAssembler* masm,
__ bind(done);
}
void ShenandoahBarrierSetAssembler::resolve_forward_pointer(MacroAssembler* masm, Register dst, Register tmp) {
assert(ShenandoahLoadRefBarrier || ShenandoahCASBarrier, "Should be enabled");
Label is_null;
__ beqz(dst, is_null);
resolve_forward_pointer_not_null(masm, dst, tmp);
__ bind(is_null);
}
// IMPORTANT: This must preserve all registers, even t0 and t1, except those explicitly
// passed in.
void ShenandoahBarrierSetAssembler::resolve_forward_pointer_not_null(MacroAssembler* masm, Register dst, Register tmp) {
assert(ShenandoahLoadRefBarrier || ShenandoahCASBarrier, "Should be enabled");
// The below loads the mark word, checks if the lowest two bits are
// set, and if so, clear the lowest two bits and copy the result
// to dst. Otherwise it leaves dst alone.
// Implementing this is surprisingly awkward. I do it here by:
// - Inverting the mark word
// - Test lowest two bits == 0
// - If so, set the lowest two bits
// - Invert the result back, and copy to dst
RegSet saved_regs = RegSet::of(t2);
bool borrow_reg = (tmp == noreg);
if (borrow_reg) {
// No free registers available. Make one useful.
tmp = t0;
if (tmp == dst) {
tmp = t1;
}
saved_regs += RegSet::of(tmp);
}
assert_different_registers(tmp, dst, t2);
__ push_reg(saved_regs, sp);
Label done;
__ ld(tmp, Address(dst, oopDesc::mark_offset_in_bytes()));
__ xori(tmp, tmp, -1); // eon with 0 is equivalent to XOR with -1
__ andi(t2, tmp, markWord::lock_mask_in_place);
__ bnez(t2, done);
__ ori(tmp, tmp, markWord::marked_value);
__ xori(dst, tmp, -1); // eon with 0 is equivalent to XOR with -1
__ bind(done);
__ pop_reg(saved_regs, sp);
}
void ShenandoahBarrierSetAssembler::load_reference_barrier(MacroAssembler* masm,
Register dst,
Address load_addr,
@ -481,90 +433,6 @@ void ShenandoahBarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembl
__ bind(done);
}
// Special Shenandoah CAS implementation that handles false negatives due
// to concurrent evacuation. The service is more complex than a
// traditional CAS operation because the CAS operation is intended to
// succeed if the reference at addr exactly matches expected or if the
// reference at addr holds a pointer to a from-space object that has
// been relocated to the location named by expected. There are two
// races that must be addressed:
// a) A parallel thread may mutate the contents of addr so that it points
// to a different object. In this case, the CAS operation should fail.
// b) A parallel thread may heal the contents of addr, replacing a
// from-space pointer held in addr with the to-space pointer
// representing the new location of the object.
// Upon entry to cmpxchg_oop, it is assured that new_val equals null
// or it refers to an object that is not being evacuated out of
// from-space, or it refers to the to-space version of an object that
// is being evacuated out of from-space.
//
// By default the value held in the result register following execution
// of the generated code sequence is 0 to indicate failure of CAS,
// non-zero to indicate success. If is_cae, the result is the value most
// recently fetched from addr rather than a boolean success indicator.
//
// Clobbers t0, t1
void ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler* masm,
Register addr,
Register expected,
Register new_val,
Assembler::Aqrl acquire,
Assembler::Aqrl release,
bool is_cae,
Register result) {
bool is_narrow = UseCompressedOops;
Assembler::operand_size size = is_narrow ? Assembler::uint32 : Assembler::int64;
assert_different_registers(addr, expected, t0, t1);
assert_different_registers(addr, new_val, t0, t1);
Label retry, success, fail, done;
__ bind(retry);
// Step1: Try to CAS.
__ cmpxchg(addr, expected, new_val, size, acquire, release, /* result */ t1);
// If success, then we are done.
__ beq(expected, t1, success);
// Step2: CAS failed, check the forwarded pointer.
__ mv(t0, t1);
if (is_narrow) {
__ decode_heap_oop(t0, t0);
}
resolve_forward_pointer(masm, t0);
__ encode_heap_oop(t0, t0);
// Report failure when the forwarded oop was not expected.
__ bne(t0, expected, fail);
// Step 3: CAS again using the forwarded oop.
__ cmpxchg(addr, t1, new_val, size, acquire, release, /* result */ t0);
// Retry when failed.
__ bne(t0, t1, retry);
__ bind(success);
if (is_cae) {
__ mv(result, expected);
} else {
__ mv(result, 1);
}
__ j(done);
__ bind(fail);
if (is_cae) {
__ mv(result, t0);
} else {
__ mv(result, zr);
}
__ bind(done);
}
void ShenandoahBarrierSetAssembler::gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
Register start, Register count, Register tmp) {
assert(ShenandoahCardBarrier, "Did you mean to enable ShenandoahCardBarrier?");
@ -792,7 +660,7 @@ void ShenandoahBarrierSetAssembler::load_c2(const MachNode* node, MacroAssembler
void ShenandoahBarrierSetAssembler::store_c2(const MachNode* node, MacroAssembler* masm, Address dst, bool dst_narrow,
Register src, bool src_narrow, Register tmp1, Register tmp2, Register tmp3) {
ShenandoahBarrierStubC2::store_pre(masm, node, tmp1, dst, tmp2, tmp3, dst_narrow);
ShenandoahBarrierStubC2::store_pre(masm, node, dst, tmp1, tmp2, tmp3, dst_narrow);
// Do the actual store
if (dst_narrow) {
@ -820,7 +688,7 @@ void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, Mac
const Assembler::Aqrl release = Assembler::rl;
const Assembler::operand_size size = narrow ? Assembler::uint32 : Assembler::int64;
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, Address(addr), tmp2, tmp3, narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, Address(addr), tmp1, tmp2, tmp3, narrow);
// CAS!
__ cmpxchg(addr, oldval, newval, size, acquire, release, /* result */ res, !exchange /* result_as_bool */);
@ -832,7 +700,7 @@ void ShenandoahBarrierSetAssembler::get_and_set_c2(const MachNode* node, MacroAs
Register newval, Register addr, Register tmp1, Register tmp2, Register tmp3, bool is_acquire) {
const bool is_narrow = node->bottom_type()->isa_narrowoop();
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp1, Address(addr, 0), tmp2, tmp3, is_narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, Address(addr, 0), tmp1, tmp2, tmp3, is_narrow);
if (is_narrow) {
if (is_acquire) {

4
src/hotspot/cpu/riscv/gc/shenandoah/shenandoahBarrierSetAssembler_riscv.hpp
View File

@ -54,8 +54,6 @@ private:
void card_barrier(MacroAssembler* masm, Register obj);
void resolve_forward_pointer(MacroAssembler* masm, Register dst, Register tmp = noreg);
void resolve_forward_pointer_not_null(MacroAssembler* masm, Register dst, Register tmp = noreg);
void load_reference_barrier(MacroAssembler* masm, Register dst, Address load_addr, DecoratorSet decorators);
void gen_write_ref_array_post_barrier(MacroAssembler* masm, DecoratorSet decorators,
@ -81,8 +79,6 @@ public:
Register obj, Register tmp, Label& slowpath);
virtual void try_peek_weak_handle_in_nmethod(MacroAssembler* masm, Register weak_handle, Register obj,
Register tmp, Label& slow_path);
void cmpxchg_oop(MacroAssembler* masm, Register addr, Register expected, Register new_val,
Assembler::Aqrl acquire, Assembler::Aqrl release, bool is_cae, Register result);
#ifdef COMPILER1
void gen_pre_barrier_stub(LIR_Assembler* ce, ShenandoahPreBarrierStub* stub);

74
src/hotspot/cpu/riscv/macroAssembler_riscv.cpp
View File

@ -593,7 +593,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
Register offset = t1;
Label L_loop_search_receiver, L_loop_search_empty;
Label L_restart, L_found_recv, L_found_empty, L_polymorphic, L_count_update;
Label L_restart, L_found_recv, L_found_empty, L_count_update;
// The code here recognizes three major cases:
// A. Fastest: receiver found in the table
@ -623,21 +623,20 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// if (receiver(i) == recv) goto found_recv(i);
// }
//
// // Fast: no receiver, but profile is full
// // Fast: no receiver, but profile is not full
// for (i = 0; i < receiver_count(); i++) {
// if (receiver(i) == null) goto found_null(i);
// }
// goto polymorphic
//
// // Slow: profile is full, polymorphic case
// count++;
// return
//
// // Slow: try to install receiver
// found_null(i):
// CAS(&receiver(i), null, recv);
// goto restart
//
// polymorphic:
// count++;
// return
//
// found_recv(i):
// *receiver_count(i)++
//
@ -654,7 +653,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
sub(t0, offset, end_receiver_offset);
bnez(t0, L_loop_search_receiver);
// Fast: no receiver, but profile is full
// Fast: no receiver, but profile is not full
mv(offset, base_receiver_offset);
bind(L_loop_search_empty);
add(t0, mdp, offset);
@ -663,9 +662,13 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
add(offset, offset, receiver_step);
sub(t0, offset, end_receiver_offset);
bnez(t0, L_loop_search_empty);
j(L_polymorphic);
// Slow: try to install receiver
// Slow: Receiver is not found and table is full.
// Increment polymorphic counter instead of receiver slot.
mv(offset, poly_count_offset);
j(L_count_update);
// Slowest: try to install receiver
bind(L_found_empty);
// Atomically swing receiver slot: null -> recv.
@ -683,16 +686,11 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// and just restart the search from the beginning.
j(L_restart);
// Counter updates:
// Increment polymorphic counter instead of receiver slot.
bind(L_polymorphic);
mv(offset, poly_count_offset);
j(L_count_update);
// Found a receiver, convert its slot offset to corresponding count offset.
bind(L_found_recv);
add(offset, offset, receiver_to_count_step);
// Finally, update the counter
bind(L_count_update);
add(t1, mdp, offset);
increment(Address(t1), DataLayout::counter_increment);
@ -4178,50 +4176,6 @@ void MacroAssembler::safepoint_poll(Label& slow_path, bool at_return, bool in_nm
}
}
void MacroAssembler::cmpxchgptr(Register oldv, Register newv, Register addr, Register tmp,
Label &succeed, Label *fail) {
assert_different_registers(addr, tmp, t0);
assert_different_registers(newv, tmp, t0);
assert_different_registers(oldv, tmp, t0);
// oldv holds comparison value
// newv holds value to write in exchange
// addr identifies memory word to compare against/update
if (UseZacas) {
mv(tmp, oldv);
atomic_cas(tmp, newv, addr, Assembler::int64, Assembler::aq, Assembler::rl);
beq(tmp, oldv, succeed);
} else {
Label retry_load, nope;
bind(retry_load);
// Load reserved from the memory location
load_reserved(tmp, addr, int64, Assembler::aqrl);
// Fail and exit if it is not what we expect
bne(tmp, oldv, nope);
// If the store conditional succeeds, tmp will be zero
store_conditional(tmp, newv, addr, int64, Assembler::rl);
beqz(tmp, succeed);
// Retry only when the store conditional failed
j(retry_load);
bind(nope);
}
// neither amocas nor lr/sc have an implied barrier in the failing case
membar(AnyAny);
mv(oldv, tmp);
if (fail != nullptr) {
j(*fail);
}
}
void MacroAssembler::cmpxchg_obj_header(Register oldv, Register newv, Register obj, Register tmp,
Label &succeed, Label *fail) {
assert(oopDesc::mark_offset_in_bytes() == 0, "assumption");
cmpxchgptr(oldv, newv, obj, tmp, succeed, fail);
}
void MacroAssembler::load_reserved(Register dst,
Register addr,
Assembler::operand_size size,

2
src/hotspot/cpu/riscv/macroAssembler_riscv.hpp
View File

@ -1203,8 +1203,6 @@ public:
#undef INSN_ENTRY_RELOC
void cmpxchg_obj_header(Register oldv, Register newv, Register obj, Register tmp, Label &succeed, Label *fail);
void cmpxchgptr(Register oldv, Register newv, Register addr, Register tmp, Label &succeed, Label *fail);
void cmpxchg(Register addr, Register expected,
Register new_val,
Assembler::operand_size size,

13
src/hotspot/cpu/s390/interp_masm_s390.cpp
View File

@ -2003,9 +2003,22 @@ void InterpreterMacroAssembler::notify_method_exit(bool native_method,
// depth. If it is possible to enter interp_only_mode we add
// the code to check if the event should be sent.
if (mode == NotifyJVMTI && (JvmtiExport::can_post_interpreter_events() || JvmtiExport::can_post_frame_pop())) {
NearLabel jvmti_post_done;
// if (thread->jvmti_thread_state() == nullptr) exit;
z_ltg(Z_R1_scratch, Address(Z_thread, JavaThread::jvmti_thread_state_offset()));
z_brz(jvmti_post_done);
// if (interp_only_mode() == false && frame_pop_cnt() == 0) exit;
z_lgf(Z_R1_scratch, Address(Z_R1_scratch, JvmtiThreadState::frame_pop_cnt_offset()));
z_o(Z_R1_scratch, Address(Z_thread, JavaThread::interp_only_mode_offset()));
z_brz(jvmti_post_done);
if (!native_method) push(state); // see frame::interpreter_frame_result()
call_VM(noreg, CAST_FROM_FN_PTR(address, InterpreterRuntime::post_method_exit));
if (!native_method) pop(state);
bind(jvmti_post_done);
}
}

152
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp
View File

@ -28,7 +28,6 @@
#include "gc/shenandoah/mode/shenandoahMode.hpp"
#include "gc/shenandoah/shenandoahBarrierSet.hpp"
#include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
#include "gc/shenandoah/shenandoahForwarding.hpp"
#include "gc/shenandoah/shenandoahHeap.inline.hpp"
#include "gc/shenandoah/shenandoahHeapRegion.hpp"
#include "gc/shenandoah/shenandoahRuntime.hpp"
@ -512,151 +511,6 @@ void ShenandoahBarrierSetAssembler::try_peek_weak_handle_in_nmethod(MacroAssembl
__ bind(done);
}
// Special Shenandoah CAS implementation that handles false negatives
// due to concurrent evacuation.
void ShenandoahBarrierSetAssembler::cmpxchg_oop(MacroAssembler* masm,
Register res, Address addr, Register oldval, Register newval,
bool exchange, Register tmp1, Register tmp2) {
assert(ShenandoahCASBarrier, "Should only be used when CAS barrier is enabled");
assert(oldval == rax, "must be in rax for implicit use in cmpxchg");
assert_different_registers(oldval, tmp1, tmp2);
assert_different_registers(newval, tmp1, tmp2);
Label L_success, L_failure;
// Remember oldval for retry logic below
if (UseCompressedOops) {
__ movl(tmp1, oldval);
} else {
__ movptr(tmp1, oldval);
}
// Step 1. Fast-path.
//
// Try to CAS with given arguments. If successful, then we are done.
if (UseCompressedOops) {
__ lock();
__ cmpxchgl(newval, addr);
} else {
__ lock();
__ cmpxchgptr(newval, addr);
}
__ jcc(Assembler::equal, L_success);
// Step 2. CAS had failed. This may be a false negative.
//
// The trouble comes when we compare the to-space pointer with the from-space
// pointer to the same object. To resolve this, it will suffice to resolve
// the value from memory -- this will give both to-space pointers.
// If they mismatch, then it was a legitimate failure.
//
// Before reaching to resolve sequence, see if we can avoid the whole shebang
// with filters.
// Filter: when offending in-memory value is null, the failure is definitely legitimate
__ testptr(oldval, oldval);
__ jcc(Assembler::zero, L_failure);
// Filter: when heap is stable, the failure is definitely legitimate
const Register thread = r15_thread;
Address gc_state(thread, in_bytes(ShenandoahThreadLocalData::gc_state_offset()));
__ testb(gc_state, ShenandoahHeap::HAS_FORWARDED);
__ jcc(Assembler::zero, L_failure);
if (UseCompressedOops) {
__ movl(tmp2, oldval);
__ decode_heap_oop(tmp2);
} else {
__ movptr(tmp2, oldval);
}
// Decode offending in-memory value.
// Test if-forwarded
__ testb(Address(tmp2, oopDesc::mark_offset_in_bytes()), markWord::marked_value);
__ jcc(Assembler::noParity, L_failure); // When odd number of bits, then not forwarded
__ jcc(Assembler::zero, L_failure); // When it is 00, then also not forwarded
// Load and mask forwarding pointer
__ movptr(tmp2, Address(tmp2, oopDesc::mark_offset_in_bytes()));
__ shrptr(tmp2, 2);
__ shlptr(tmp2, 2);
if (UseCompressedOops) {
__ decode_heap_oop(tmp1); // decode for comparison
}
// Now we have the forwarded offender in tmp2.
// Compare and if they don't match, we have legitimate failure
__ cmpptr(tmp1, tmp2);
__ jcc(Assembler::notEqual, L_failure);
// Step 3. Need to fix the memory ptr before continuing.
//
// At this point, we have from-space oldval in the register, and its to-space
// address is in tmp2. Let's try to update it into memory. We don't care if it
// succeeds or not. If it does, then the retrying CAS would see it and succeed.
// If this fixup fails, this means somebody else beat us to it, and necessarily
// with to-space ptr store. We still have to do the retry, because the GC might
// have updated the reference for us.
if (UseCompressedOops) {
__ encode_heap_oop(tmp2); // previously decoded at step 2.
}
if (UseCompressedOops) {
__ lock();
__ cmpxchgl(tmp2, addr);
} else {
__ lock();
__ cmpxchgptr(tmp2, addr);
}
// Step 4. Try to CAS again.
//
// This is guaranteed not to have false negatives, because oldval is definitely
// to-space, and memory pointer is to-space as well. Nothing is able to store
// from-space ptr into memory anymore. Make sure oldval is restored, after being
// garbled during retries.
//
if (UseCompressedOops) {
__ movl(oldval, tmp2);
} else {
__ movptr(oldval, tmp2);
}
if (UseCompressedOops) {
__ lock();
__ cmpxchgl(newval, addr);
} else {
__ lock();
__ cmpxchgptr(newval, addr);
}
if (!exchange) {
__ jccb(Assembler::equal, L_success); // fastpath, peeking into Step 5, no need to jump
}
// Step 5. If we need a boolean result out of CAS, set the flag appropriately.
// and promote the result. Note that we handle the flag from both the 1st and 2nd CAS.
// Otherwise, failure witness for CAE is in oldval on all paths, and we can return.
if (exchange) {
__ bind(L_failure);
__ bind(L_success);
} else {
assert(res != noreg, "need result register");
Label exit;
__ bind(L_failure);
__ xorptr(res, res);
__ jmpb(exit);
__ bind(L_success);
__ movptr(res, 1);
__ bind(exit);
}
}
#ifdef PRODUCT
#define BLOCK_COMMENT(str) /* nothing */
#else
@ -926,7 +780,7 @@ void ShenandoahBarrierSetAssembler::store_c2(const MachNode* node, MacroAssemble
Register src, bool src_narrow,
Register tmp) {
ShenandoahBarrierStubC2::store_pre(masm, node, tmp, dst, noreg, noreg, dst_narrow);
ShenandoahBarrierStubC2::store_pre(masm, node, dst, tmp, noreg, noreg, dst_narrow);
// Need to encode into tmp, because we cannot clobber src.
if (dst_narrow && !src_narrow) {
@ -961,7 +815,7 @@ void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, Mac
assert_different_registers(oldval, tmp, addr.base(), addr.index());
assert_different_registers(newval, tmp, addr.base(), addr.index());
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp, addr, noreg, noreg, narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, tmp, noreg, noreg, narrow);
// CAS!
__ lock();
@ -982,7 +836,7 @@ void ShenandoahBarrierSetAssembler::compare_and_set_c2(const MachNode* node, Mac
void ShenandoahBarrierSetAssembler::get_and_set_c2(const MachNode* node, MacroAssembler* masm, Register newval, Address addr, Register tmp, bool narrow) {
assert_different_registers(newval, tmp, addr.base(), addr.index());
ShenandoahBarrierStubC2::load_store_pre(masm, node, tmp, addr, noreg, noreg, narrow);
ShenandoahBarrierStubC2::load_store_pre(masm, node, addr, tmp, noreg, noreg, narrow);
if (narrow) {
__ xchgl(newval, addr);

3
src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.hpp
View File

@ -60,9 +60,6 @@ public:
void load_reference_barrier(MacroAssembler* masm, Register dst, Address src, DecoratorSet decorators);
void cmpxchg_oop(MacroAssembler* masm,
Register res, Address addr, Register oldval, Register newval,
bool exchange, Register tmp1, Register tmp2);
virtual void arraycopy_prologue(MacroAssembler* masm, DecoratorSet decorators, BasicType type,
Register src, Register dst, Register count);
virtual void arraycopy_epilogue(MacroAssembler* masm, DecoratorSet decorators, BasicType type,

31
src/hotspot/cpu/x86/macroAssembler_x86.cpp
View File

@ -4901,7 +4901,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
Register offset = rscratch1;
Label L_loop_search_receiver, L_loop_search_empty;
Label L_restart, L_found_recv, L_found_empty, L_polymorphic, L_count_update;
Label L_restart, L_found_recv, L_found_empty, L_count_update;
// The code here recognizes three major cases:
// A. Fastest: receiver found in the table
@ -4931,21 +4931,20 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// if (receiver(i) == recv) goto found_recv(i);
// }
//
// // Fast: no receiver, but profile is full
// // Fast: no receiver, but profile is not full
// for (i = 0; i < receiver_count(); i++) {
// if (receiver(i) == null) goto found_null(i);
// }
// goto polymorphic
//
// // Slow: profile is full, polymorphic case
// count++;
// return
//
// // Slow: try to install receiver
// found_null(i):
// CAS(&receiver(i), null, recv);
// goto restart
//
// polymorphic:
// count++;
// return
//
// found_recv(i):
// *receiver_count(i)++
//
@ -4961,7 +4960,7 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
cmpptr(offset, end_receiver_offset);
jccb(Assembler::notEqual, L_loop_search_receiver);
// Fast: no receiver, but profile is full
// Fast: no receiver, but profile is not full
movptr(offset, base_receiver_offset);
bind(L_loop_search_empty);
cmpptr(Address(mdp, offset, Address::times_ptr), NULL_WORD);
@ -4969,9 +4968,13 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
addptr(offset, receiver_step);
cmpptr(offset, end_receiver_offset);
jccb(Assembler::notEqual, L_loop_search_empty);
jmpb(L_polymorphic);
// Slow: try to install receiver
// Slow: Receiver is not found and table is full.
// Increment polymorphic counter instead of receiver slot.
movptr(offset, poly_count_offset);
jmpb(L_count_update);
// Slowest: try to install receiver
bind(L_found_empty);
// Atomically swing receiver slot: null -> recv.
@ -5023,17 +5026,11 @@ void MacroAssembler::profile_receiver_type(Register recv, Register mdp, int mdp_
// and just restart the search from the beginning.
jmpb(L_restart);
// Counter updates:
// Increment polymorphic counter instead of receiver slot.
bind(L_polymorphic);
movptr(offset, poly_count_offset);
jmpb(L_count_update);
// Found a receiver, convert its slot offset to corresponding count offset.
bind(L_found_recv);
addptr(offset, receiver_to_count_step);
// Finally, update the counter
bind(L_count_update);
addptr(Address(mdp, offset, Address::times_ptr), DataLayout::counter_increment);
}

5
src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
View File

@ -4904,6 +4904,11 @@ void StubGenerator::generate_compiler_stubs() {
StubRoutines::_intpoly_assign = generate_intpoly_assign();
}
if (UseIntPoly25519Intrinsics) {
StubRoutines::_intpoly_mult_25519 = generate_intpoly_mult_25519();
StubRoutines::_intpoly_square_25519 = generate_intpoly_square_25519();
}
if (UseMD5Intrinsics) {
StubRoutines::_md5_implCompress = generate_md5_implCompress(StubId::stubgen_md5_implCompress_id);
StubRoutines::_md5_implCompressMB = generate_md5_implCompress(StubId::stubgen_md5_implCompressMB_id);

3
src/hotspot/cpu/x86/stubGenerator_x86_64.hpp
View File

@ -496,6 +496,9 @@ class StubGenerator: public StubCodeGenerator {
address generate_intpoly_montgomeryMult_P256();
address generate_intpoly_assign();
address generate_intpoly_mult_25519();
address generate_intpoly_square_25519();
// SHA3 stubs
void generate_sha3_stubs();

306
src/hotspot/cpu/x86/stubGenerator_x86_64_poly25519.cpp Normal file
View File

@ -0,0 +1,306 @@
/*
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation. Oracle designates this
* particular file as subject to the "Classpath" exception as provided
* by Oracle in the LICENSE file that accompanied this code.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*/
#include "macroAssembler_x86.hpp"
#include "stubGenerator_x86_64.hpp"
#define __ _masm->
const int32_t term = 19;
const int32_t limbs = 5;
const int32_t bpl = 51;
const int32_t rem = 64 - bpl;
const uint64_t MASK = 0x7FFFFFFFFFFFF;
const uint64_t CARRY_ADD = 0x4000000000000;
// Multiplication operation for polynomial arithmetic in Curve25519.
//
// This is the same algorithm as used in Java, except we use pseudo-Mersenne
// reduction to reduce register pressure instead of using the full 10 columns
// in Java.
void multiply_25519_scalar(const Register aLimbs, const Register bLimbs, const Register rLimbs, Register c[], Register bArg, Register d, Register b, Register mask, MacroAssembler* _masm) {
for (int i = 0; i < limbs; i++) {
__ xorq(c[i], c[i]);
}
__ mov64(mask, MASK);
__ movq(bArg, bLimbs);
// Perform high/low multiplication with signed 5x51 bit limbs
for (int i = 0; i < limbs; i++) {
__ movq(b, Address(bArg, i * 8));
for (int j = 0; j < limbs; j++) {
__ movq(rax, Address(aLimbs, j * 8));
__ imulq(b); // rdx:rax = a * b
__ movq(d, rax);
__ andq(d, mask);
__ shrq(rax, bpl);
__ shlq(rdx, rem);
__ orq(rax, rdx);
// Fold in pseudo-Mersenne reduction
if ((i + j + 1) >= limbs) {
__ imulq(rax, rax, term);
}
if ((i + j) >= limbs) {
__ imulq(d, d, term);
}
__ addq(c[(i + j) % limbs], d);
__ addq(c[(i + j + 1) % limbs], rax);
}
}
// Carry-add with reduction from high limb
Register carry = bArg;
__ mov64(mask, CARRY_ADD);
__ movq(carry, mask);
// Limb 3
__ addq(carry, c[3]);
__ sarq(carry, bpl);
__ addq(c[4], carry);
__ shlq(carry, bpl);
__ subq(c[3], carry);
// Limb 4
__ movq(carry, mask);
__ addq(carry, c[4]);
__ sarq(carry, bpl);
// Reduce high order limb and fold back into low order limb
__ mov64(rax, term);
__ imulq(carry);
__ addq(c[0], rax);
__ shlq(carry, bpl);
__ subq(c[4], carry);
// Limbs 0 - 3
for (int i = 0; i < (limbs - 1); i++) {
__ movq(carry, mask);
__ addq(carry, c[i]);
__ sarq(carry, bpl);
__ addq(c[i + 1], carry);
__ shlq(carry, bpl);
__ subq(c[i], carry);
}
__ pop_ppx(rdx);
for (int i = 0; i < limbs; i++) {
__ movq(Address(rLimbs, i * 8), c[i]);
}
}
// Squaring operation for polynomial arithmetic in Curve25519.
//
// This is the same algorithm as used in Java, except we use pseudo-Mersenne
// reduction to reduce register pressure instead of using the full 10 columns
// in Java.
void square_25519_scalar(const Register aLimbs, const Register rLimbs, Register c[], Register aArg, Register d, Register carry, Register mask, MacroAssembler* _masm) {
for (int i = 0; i < limbs; i++) {
__ xorq(c[i], c[i]);
}
__ mov64(mask, MASK);
// Perform high/low multiplication with signed 5x51 bit limbs
for (int i = 0; i < limbs; i++) {
__ movq(aArg, Address(aLimbs, i * 8));
__ movq(rax, aArg);
__ imulq(aArg); // rdx:rax = a[j] * a[i]
__ movq(d, rax);
__ andq(d, mask);
__ shrq(rax, bpl);
__ shlq(rdx, rem);
__ orq(rax, rdx); // rax = dd
if ((i * 2 + 1) >= limbs) {
__ imulq(rax, rax, term);
}
if ((i * 2) >= limbs) {
__ imulq(d, d, term);
}
__ addq(c[(i * 2) % limbs], d);
__ addq(c[(i * 2 + 1) % limbs], rax);
for (int j = i + 1; j < limbs; j++) {
__ movq(rax, Address(aLimbs, j * 8));
__ imulq(aArg); // rdx:rax = a * a
__ movq(d, rax);
__ andq(d, mask);
__ shlq(d, 1);
__ shrq(rax, bpl);
__ shlq(rdx, rem);
__ orq(rax, rdx); // rax = dd
__ shlq(rax, 1);
if ((j + i + 1) >= limbs) {
__ imulq(rax, rax, term);
}
if ((j + i) >= limbs) {
__ imulq(d, d, term);
}
__ addq(c[(i + j) % limbs], d);
__ addq(c[(i + j + 1) % limbs], rax);
}
}
// Carry-add with reduction from high limb
// Limb 3
__ mov64(mask, CARRY_ADD);
__ movq(carry, mask);
__ addq(carry, c[3]);
__ sarq(carry, bpl);
__ addq(c[4], carry);
__ shlq(carry, bpl);
__ subq(c[3], carry);
// Limb 4
__ movq(carry, mask);
__ addq(carry, c[4]);
__ sarq(carry, bpl);
// Reduce high order limb and fold back into low order limb
__ mov64(rax, term);
__ imulq(carry);
__ addq(c[0], rax);
__ shlq(carry, bpl);
__ subq(c[4], carry);
// Limbs 0 - 3
for (int i = 0; i < (limbs - 1); i++) {
__ movq(carry, mask);
__ addq(carry, c[i]);
__ sarq(carry, bpl);
__ addq(c[i + 1], carry);
__ shlq(carry, bpl);
__ subq(c[i], carry);
}
__ pop_ppx(rdx);
for (int i = 0; i < limbs; i++) {
__ movq(Address(rLimbs, i * 8), c[i]);
}
}
address StubGenerator::generate_intpoly_mult_25519() {
StubId stub_id = StubId::stubgen_intpoly_mult_25519_id;
int entry_count = StubInfo::entry_count(stub_id);
assert(entry_count == 1, "sanity check");
address start = load_archive_data(stub_id);
if (start != nullptr) {
return start;
}
__ align(CodeEntryAlignment);
StubCodeMark mark(this, stub_id);
start = __ pc();
__ enter();
// Register Map
const Register aLimbs = c_rarg0; // rdi | rcx
const Register bLimbs = c_rarg1; // rsi | rdx
const Register rLimbs = c_rarg2; // rdx | r8
Register c[] = {r9, r10, r11, r12, r13};
Register bArg = r14;
Register d = r15;
Register b = rbp;
Register mask = rbx;
__ push_ppx(rbp);
__ push_ppx(rbx);
__ push_ppx(r12);
__ push_ppx(r13);
__ push_ppx(r14);
__ push_ppx(r15);
__ push_ppx(rdx);
multiply_25519_scalar(aLimbs, bLimbs, rLimbs, c, bArg, d, b, mask, _masm);
// __ pop_ppx(rdx); // restored in the helper already
__ pop_ppx(r15);
__ pop_ppx(r14);
__ pop_ppx(r13);
__ pop_ppx(r12);
__ pop_ppx(rbx);
__ pop_ppx(rbp);
__ leave();
__ ret(0);
// Record the stub entry and end
store_archive_data(stub_id, start, __ pc());
return start;
}
address StubGenerator::generate_intpoly_square_25519() {
StubId stub_id = StubId::stubgen_intpoly_square_25519_id;
int entry_count = StubInfo::entry_count(stub_id);
assert(entry_count == 1, "sanity check");
address start = load_archive_data(stub_id);
if (start != nullptr) {
return start;
}
__ align(CodeEntryAlignment);
StubCodeMark mark(this, stub_id);
start = __ pc();
__ enter();
// Register Map
const Register aLimbs = c_rarg0; // rdi | rcx
const Register rLimbs = c_rarg1; // rsi | rdx
Register c[] = {r9, r10, r11, r12, r13};
Register aArg = r14;
Register d = r15;
Register carry = rbp;
Register mask = rbx;
__ push_ppx(rbp);
__ push_ppx(rbx);
__ push_ppx(r12);
__ push_ppx(r13);
__ push_ppx(r14);
__ push_ppx(r15);
__ push_ppx(rdx);
square_25519_scalar(aLimbs, rLimbs, c, aArg, d, carry, mask, _masm);
// __ pop_ppx(rdx); // restored in the helper already
__ pop_ppx(r15);
__ pop_ppx(r14);
__ pop_ppx(r13);
__ pop_ppx(r12);
__ pop_ppx(rbx);
__ pop_ppx(rbp);
__ leave();
__ ret(0);
// Record the stub entry and end
store_archive_data(stub_id, start, __ pc());
return start;
}
#undef __

4
src/hotspot/cpu/x86/stubGenerator_x86_64_poly_mont.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2024, 2025, Intel Corporation. All rights reserved.
* Copyright (c) 2024, 2026, Intel Corporation. All rights reserved.
*
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -676,7 +676,7 @@ address StubGenerator::generate_intpoly_assign() {
// KNOWN Lengths:
// MontgomeryIntPolynP256: 5 = 4 + 1
// IntegerPolynomial1305: 5 = 4 + 1
// IntegerPolynomial25519: 10 = 8 + 2
// IntegerPolynomial25519: 5 = 4 + 1
// IntegerPolynomialP256: 10 = 8 + 2
// Curve25519OrderField: 10 = 8 + 2
// Curve25519OrderField: 10 = 8 + 2

4
src/hotspot/cpu/x86/vm_version_x86.cpp
View File

@ -1407,6 +1407,10 @@ void VM_Version::get_processor_features() {
FLAG_SET_DEFAULT(UseIntPolyIntrinsics, false);
}
if (FLAG_IS_DEFAULT(UseIntPoly25519Intrinsics)) {
UseIntPoly25519Intrinsics = true;
}
if (FLAG_IS_DEFAULT(UseMultiplyToLenIntrinsic)) {
UseMultiplyToLenIntrinsic = true;
}

2
src/hotspot/cpu/x86/x86.ad
View File

@ -13959,6 +13959,7 @@ instruct orL_rReg_ndd(rRegL dst, rRegL src1, rRegL src2, rFlagsReg cr)
// Use any_RegP to match R15 (TLS register) without spilling.
instruct orL_rReg_castP2X(rRegL dst, any_RegP src, rFlagsReg cr) %{
predicate(!UseAPX);
match(Set dst (OrL dst (CastP2X src)));
effect(KILL cr);
flag(PD::Flag_sets_sign_flag, PD::Flag_sets_zero_flag, PD::Flag_sets_parity_flag, PD::Flag_clears_overflow_flag, PD::Flag_clears_carry_flag);
@ -13971,6 +13972,7 @@ instruct orL_rReg_castP2X(rRegL dst, any_RegP src, rFlagsReg cr) %{
%}
instruct orL_rReg_castP2X_ndd(rRegL dst, any_RegP src1, any_RegP src2, rFlagsReg cr) %{
predicate(UseAPX);
match(Set dst (OrL src1 (CastP2X src2)));
effect(KILL cr);
flag(PD::Flag_sets_sign_flag, PD::Flag_sets_zero_flag, PD::Flag_sets_parity_flag, PD::Flag_clears_overflow_flag, PD::Flag_clears_carry_flag);

36
src/hotspot/os/posix/os_posix.cpp
View File

@ -167,11 +167,11 @@ void os::check_core_dump_prerequisites(char* buffer, size_t bufferSize, bool che
}
}
bool os::committed_in_range(address start, size_t size, address& committed_start, size_t& committed_size) {
bool os::first_resident_in_range(address start, size_t size, address& resident_start, size_t& resident_size) {
#ifdef _AIX
committed_start = start;
committed_size = size;
resident_start = start;
resident_size = size;
return true;
#else
@ -188,10 +188,10 @@ bool os::committed_in_range(address start, size_t size, address& committed_start
assert(is_aligned(start, page_sz), "Start address must be page aligned");
assert(is_aligned(size, page_sz), "Size must be page aligned");
committed_start = nullptr;
resident_start = nullptr;
int loops = checked_cast<int>((pages + stripe - 1) / stripe);
int committed_pages = 0;
int resident_pages = 0;
address loop_base = start;
bool found_range = false;
@ -210,7 +210,7 @@ bool os::committed_in_range(address start, size_t size, address& committed_start
// During shutdown, some memory goes away without properly notifying NMT,
// E.g. ConcurrentGCThread/WatcherThread can exit without deleting thread object.
// Bailout and return as not committed for now.
// Bailout and return as not resident for now.
if (mincore_return_value == -1 && errno == ENOMEM) {
return false;
}
@ -224,32 +224,32 @@ bool os::committed_in_range(address start, size_t size, address& committed_start
assert(mincore_return_value == 0, "Range must be valid");
// Process this stripe
for (uintx vecIdx = 0; vecIdx < pages_to_query; vecIdx ++) {
if ((vec[vecIdx] & 0x01) == 0) { // not committed
if ((vec[vecIdx] & 0x01) == 0) { // not resident
// End of current contiguous region
if (committed_start != nullptr) {
if (resident_start != nullptr) {
found_range = true;
break;
}
} else { // committed
} else { // resident
// Start of region
if (committed_start == nullptr) {
committed_start = loop_base + page_sz * vecIdx;
if (resident_start == nullptr) {
resident_start = loop_base + page_sz * vecIdx;
}
committed_pages ++;
resident_pages ++;
}
}
loop_base += pages_to_query * page_sz;
}
if (committed_start != nullptr) {
assert(committed_pages > 0, "Must have committed region");
assert(committed_pages <= int(size / page_sz), "Can not commit more than it has");
assert(committed_start >= start && committed_start < start + size, "Out of range");
committed_size = page_sz * committed_pages;
if (resident_start != nullptr) {
assert(resident_pages > 0, "Must have a resident region");
assert(resident_pages <= int(size / page_sz), "Resident size exceeds region size");
assert(resident_start >= start && resident_start < start + size, "Out of range");
resident_size = page_sz * resident_pages;
return true;
} else {
assert(committed_pages == 0, "Should not have committed region");
assert(resident_pages == 0, "Should not have a resident region");
return false;
}
#endif

276
src/hotspot/os/windows/os_windows.cpp
View File

@ -243,6 +243,16 @@ static LPVOID virtualAllocExNuma(HANDLE hProcess, LPVOID lpAddress, SIZE_T dwSiz
return result;
}
void* os::win32::lookup_kernelbase_symbol(const char* name) {
// Pass a small ebuf so dll_load logs failures, but don't use it here to avoid redundancy.
char ebuf[1024];
static void* const handle = os::dll_load("KernelBase", ebuf, sizeof(ebuf));
if (handle == nullptr) {
return nullptr;
}
return os::dll_lookup(handle, name);
}
// Logging wrapper for MapViewOfFileEx
static LPVOID mapViewOfFileEx(HANDLE hFileMappingObject, DWORD dwDesiredAccess, DWORD dwFileOffsetHigh,
DWORD dwFileOffsetLow, SIZE_T dwNumberOfBytesToMap, LPVOID lpBaseAddress) {
@ -465,36 +475,63 @@ void os::current_stack_base_and_size(address* stack_base, size_t* stack_size) {
*stack_size = size;
}
bool os::committed_in_range(address start, size_t size, address& committed_start, size_t& committed_size) {
MEMORY_BASIC_INFORMATION minfo;
committed_start = nullptr;
committed_size = 0;
address top = start + size;
const address start_addr = start;
while (start < top) {
VirtualQuery(start, &minfo, sizeof(minfo));
if ((minfo.State & MEM_COMMIT) == 0) { // not committed
if (committed_start != nullptr) {
break;
}
} else { // committed
if (committed_start == nullptr) {
committed_start = start;
}
size_t offset = start - (address)minfo.BaseAddress;
committed_size += minfo.RegionSize - offset;
}
start = (address)minfo.BaseAddress + minfo.RegionSize;
}
bool os::first_resident_in_range(address start, size_t size, address& resident_start, size_t& resident_size) {
constexpr size_t stripe = 1024; // query this many pages each time
PSAPI_WORKING_SET_EX_INFORMATION wsinfo[stripe];
if (committed_start == nullptr) {
assert(committed_size == 0, "Sanity");
return false;
} else {
assert(committed_start >= start_addr && committed_start < top, "Out of range");
// current region may go beyond the limit, trim to the limit
committed_size = MIN2(committed_size, size_t(top - committed_start));
size_t page_sz = os::vm_page_size();
uintx pages_left = size / page_sz;
assert(is_aligned(start, page_sz), "Start address must be page aligned");
assert(is_aligned(size, page_sz), "Size must be page aligned");
resident_start = nullptr;
uintx loops = (pages_left + stripe - 1) / stripe;
uintx resident_pages = 0;
address pos = start;
bool found_range = false;
for (uintx index = 0; index < loops && !found_range; index++) {
assert(pages_left > 0, "Nothing to do");
uintx pages_to_query = MIN2(pages_left, stripe);
pages_left -= pages_to_query;
for (uintx i = 0; i < pages_to_query; i++) {
wsinfo[i].VirtualAddress = (PVOID)(pos + i * page_sz);
}
BOOL success = QueryWorkingSetEx(GetCurrentProcess(), wsinfo, pages_to_query * sizeof(PSAPI_WORKING_SET_EX_INFORMATION));
if (!success) {
return false;
}
for (uintx i = 0; i < pages_to_query; i++) {
if (wsinfo[i].VirtualAttributes.Valid == 0) {
if (resident_start != nullptr) {
found_range = true;
break;
}
// Still searching for start of resident region
} else {
if (resident_start == nullptr) {
// Found first resident page in region
resident_start = pos + i * page_sz;
}
resident_pages++;
}
}
pos += pages_to_query * page_sz;
}
if (resident_start != nullptr) {
assert(resident_pages > 0, "Must have a resident region");
assert(resident_pages <= size / page_sz, "Resident size exceeds region size");
assert(resident_start >= start && resident_start < start + size, "Out of range");
resident_size = page_sz * resident_pages;
return true;
} else {
assert(resident_pages == 0, "Should not have a resident region");
return false;
}
}
@ -3223,9 +3260,9 @@ char* os::map_memory_to_file(char* base, size_t size, int fd) {
assert(fd != -1, "File descriptor is not valid");
HANDLE fh = (HANDLE)_get_osfhandle(fd);
HANDLE fileMapping = CreateFileMapping(fh, nullptr, PAGE_READWRITE,
HANDLE file_mapping = CreateFileMapping(fh, nullptr, PAGE_READWRITE,
(DWORD)(size >> 32), (DWORD)(size & 0xFFFFFFFF), nullptr);
if (fileMapping == nullptr) {
if (file_mapping == nullptr) {
if (GetLastError() == ERROR_DISK_FULL) {
vm_exit_during_initialization(err_msg("Could not allocate sufficient disk space for Java heap"));
}
@ -3236,9 +3273,9 @@ char* os::map_memory_to_file(char* base, size_t size, int fd) {
return nullptr;
}
LPVOID addr = mapViewOfFileEx(fileMapping, FILE_MAP_WRITE, 0, 0, size, base);
LPVOID addr = mapViewOfFileEx(file_mapping, FILE_MAP_WRITE, 0, 0, size, base);
CloseHandle(fileMapping);
CloseHandle(file_mapping);
return (char*)addr;
}
@ -3251,40 +3288,75 @@ char* os::replace_existing_mapping_with_file_mapping(char* base, size_t size, in
return map_memory_to_file(base, size, fd);
}
// VirtualAlloc2 / MapViewOfFile3 (Windows 1803+). Resolved in os::init_2() via lookup_kernelbase_symbol.
os::win32::VirtualAlloc2Fn os::win32::VirtualAlloc2 = nullptr;
os::win32::MapViewOfFile3Fn os::win32::MapViewOfFile3 = nullptr;
static bool is_VirtualAlloc2_supported() {
return os::win32::VirtualAlloc2 != nullptr;
}
static bool is_MapViewOfFile3_supported() {
return os::win32::MapViewOfFile3 != nullptr;
}
// Multiple threads can race in this code but it's not possible to unmap small sections of
// virtual space to get requested alignment, like posix-like os's.
// Windows prevents multiple thread from remapping over each other so this loop is thread-safe.
static char* map_or_reserve_memory_aligned(size_t size, size_t alignment, int file_desc, MemTag mem_tag) {
static char* reserve_memory_aligned(size_t size, size_t alignment, MemTag mem_tag) {
assert(is_aligned(alignment, os::vm_allocation_granularity()),
"Alignment must be a multiple of allocation granularity (page size)");
"Alignment must be a multiple of allocation granularity");
assert(is_aligned(size, os::vm_allocation_granularity()),
"Size must be a multiple of allocation granularity (page size)");
"Size must be a multiple of allocation granularity");
size_t extra_size = size + alignment;
assert(extra_size >= size, "overflow, size is too large to allow alignment");
char* aligned_base = nullptr;
static const int max_attempts = 20;
constexpr int max_attempts = 20;
for (int attempt = 0; attempt < max_attempts && aligned_base == nullptr; attempt ++) {
char* extra_base = file_desc != -1 ? os::map_memory_to_file(extra_size, file_desc, mem_tag) :
os::reserve_memory(extra_size, mem_tag);
char* extra_base = os::reserve_memory(extra_size, mem_tag);
if (extra_base == nullptr) {
return nullptr;
}
// Do manual alignment
aligned_base = align_up(extra_base, alignment);
os::release_memory(extra_base, extra_size);
if (file_desc != -1) {
os::unmap_memory(extra_base, extra_size);
} else {
os::release_memory(extra_base, extra_size);
// A racing thread may have taken this region instead of us, which is why we loop and retry.
aligned_base = os::attempt_reserve_memory_at(aligned_base, size, mem_tag);
}
assert(aligned_base != nullptr,
"Did not manage to reserve after %d attempts (size %zu, alignment %zu)", max_attempts, size, alignment);
return aligned_base;
}
// Similar to reserve_memory_aligned, other reservation/mapping requests can race with this function.
static char* map_memory_aligned(size_t size, size_t alignment, int file_desc, MemTag mem_tag) {
assert(is_aligned(alignment, os::vm_allocation_granularity()),
"Alignment must be a multiple of allocation granularity");
assert(is_aligned(size, os::vm_allocation_granularity()),
"Size must be a multiple of allocation granularity");
size_t extra_size = size + alignment;
assert(extra_size >= size, "overflow, size is too large to allow alignment");
char* aligned_base = nullptr;
constexpr int max_attempts = 20;
for (int attempt = 0; attempt < max_attempts && aligned_base == nullptr; attempt ++) {
char* extra_base = os::map_memory_to_file(extra_size, file_desc, mem_tag);
if (extra_base == nullptr) {
return nullptr;
}
aligned_base = align_up(extra_base, alignment);
os::unmap_memory(extra_base, extra_size);
// Attempt to map, into the just vacated space, the slightly smaller aligned area.
// Which may fail, hence the loop.
aligned_base = file_desc != -1 ? os::attempt_map_memory_to_file_at(aligned_base, size, file_desc, mem_tag) :
os::attempt_reserve_memory_at(aligned_base, size, mem_tag);
// A racing thread may have taken this region instead of us, which is why we loop and retry.
aligned_base = os::attempt_map_memory_to_file_at(aligned_base, size, file_desc, mem_tag);
}
assert(aligned_base != nullptr,
@ -3293,6 +3365,84 @@ static char* map_or_reserve_memory_aligned(size_t size, size_t alignment, int fi
return aligned_base;
}
// MapViewOfFile3 supports alignment natively.
static char* map_memory_aligned_va2(size_t size, size_t alignment, int file_desc, MemTag mem_tag) {
assert(file_desc != -1, "File descriptor should not be -1");
assert(is_aligned(alignment, os::vm_allocation_granularity()),
"Alignment must be a multiple of allocation granularity");
assert(is_aligned(size, os::vm_allocation_granularity()),
"Size must be a multiple of allocation granularity");
MEM_ADDRESS_REQUIREMENTS requirements = {0};
requirements.Alignment = alignment;
MEM_EXTENDED_PARAMETER param = {0};
param.Type = MemExtendedParameterAddressRequirements;
param.Pointer = &requirements;
char* aligned_base = nullptr;
// File-backed aligned mapping.
HANDLE fh = (HANDLE)_get_osfhandle(file_desc);
HANDLE file_mapping = CreateFileMapping(fh, nullptr, PAGE_READWRITE,(DWORD)(size >> 32), (DWORD)(size & 0xFFFFFFFF), nullptr);
DWORD err = GetLastError();
if (file_mapping != nullptr) {
aligned_base = (char*)os::win32::MapViewOfFile3(
file_mapping,
GetCurrentProcess(),
nullptr, // let the system choose an aligned address
0, // offset
size,
0, // no special allocation type flags
PAGE_READWRITE,
&param, 1);
err = GetLastError();
CloseHandle(file_mapping);
}
if (aligned_base != nullptr) {
assert(is_aligned(aligned_base, alignment), "Result must be aligned");
MemTracker::record_virtual_memory_reserve_and_commit(aligned_base, size, CALLER_PC, mem_tag);
return aligned_base;
}
log_trace(os)("Aligned allocation via MapViewOfFile3 failed, falling back to retry loop. GetLastError->%lu.", err);
return map_memory_aligned(size, alignment, file_desc, mem_tag);
}
// VirtualAlloc2 supports alignment natively.
static char* reserve_memory_aligned_va2(size_t size, size_t alignment, MemTag mem_tag) {
assert(is_aligned(alignment, os::vm_allocation_granularity()),
"Alignment must be a multiple of allocation granularity");
assert(is_aligned(size, os::vm_allocation_granularity()),
"Size must be a multiple of allocation granularity");
MEM_ADDRESS_REQUIREMENTS requirements = {0};
requirements.Alignment = alignment;
MEM_EXTENDED_PARAMETER param = {0};
param.Type = MemExtendedParameterAddressRequirements;
param.Pointer = &requirements;
char* aligned_base = nullptr;
// Anonymous aligned reservation.
aligned_base = (char*)os::win32::VirtualAlloc2(
GetCurrentProcess(),
nullptr, // let the system choose an aligned address
size,
MEM_RESERVE,
PAGE_READWRITE,
&param, 1);
if (aligned_base != nullptr) {
assert(is_aligned(aligned_base, alignment), "Result must be aligned");
MemTracker::record_virtual_memory_reserve(aligned_base, size, CALLER_PC, mem_tag);
return aligned_base;
}
log_trace(os)("Aligned allocation via VirtualAlloc2 failed, falling back to retry loop. GetLastError->%lu.", GetLastError());
return reserve_memory_aligned(size, alignment, mem_tag);
}
size_t os::commit_memory_limit() {
BOOL is_in_job_object = false;
BOOL res = IsProcessInJob(GetCurrentProcess(), nullptr, &is_in_job_object);
@ -3340,11 +3490,17 @@ size_t os::reserve_memory_limit() {
char* os::reserve_memory_aligned(size_t size, size_t alignment, MemTag mem_tag, bool exec) {
// exec can be ignored
return map_or_reserve_memory_aligned(size, alignment, -1/* file_desc */, mem_tag);
if (is_VirtualAlloc2_supported()) {
return reserve_memory_aligned_va2(size, alignment, mem_tag);
}
return reserve_memory_aligned(size, alignment, mem_tag);
}
char* os::map_memory_to_file_aligned(size_t size, size_t alignment, int fd, MemTag mem_tag) {
return map_or_reserve_memory_aligned(size, alignment, fd, mem_tag);
if (is_MapViewOfFile3_supported()) {
return map_memory_aligned_va2(size, alignment, fd, mem_tag);
}
return map_memory_aligned(size, alignment, fd, mem_tag);
}
char* os::pd_reserve_memory(size_t bytes, bool exec) {
@ -4561,21 +4717,21 @@ jint os::init_2(void) {
// Lookup SetThreadDescription - the docs state we must use runtime-linking of
// kernelbase.dll, so that is what we do.
HINSTANCE _kernelbase = LoadLibrary(TEXT("kernelbase.dll"));
if (_kernelbase != nullptr) {
_SetThreadDescription =
reinterpret_cast<SetThreadDescriptionFnPtr>(
GetProcAddress(_kernelbase,
"SetThreadDescription"));
_SetThreadDescription = reinterpret_cast<SetThreadDescriptionFnPtr>(
os::win32::lookup_kernelbase_symbol("SetThreadDescription"));
#ifdef ASSERT
_GetThreadDescription =
reinterpret_cast<GetThreadDescriptionFnPtr>(
GetProcAddress(_kernelbase,
"GetThreadDescription"));
_GetThreadDescription = reinterpret_cast<GetThreadDescriptionFnPtr>(
os::win32::lookup_kernelbase_symbol("GetThreadDescription"));
#endif
}
log_info(os, thread)("The SetThreadDescription API is%s available.", _SetThreadDescription == nullptr ? " not" : "");
// Prepare KernelBase APIs (VirtualAlloc2, MapViewOfFile3) if available (Windows version 1803).
os::win32::VirtualAlloc2 = reinterpret_cast<os::win32::VirtualAlloc2Fn>(
os::win32::lookup_kernelbase_symbol("VirtualAlloc2"));
os::win32::MapViewOfFile3 = reinterpret_cast<os::win32::MapViewOfFile3Fn>(
os::win32::lookup_kernelbase_symbol("MapViewOfFile3"));
log_debug(os)("VirtualAlloc2 is%s available.", os::win32::VirtualAlloc2 == nullptr ? " not" : "");
log_debug(os)("MapViewOfFile3 is%s available.", os::win32::MapViewOfFile3 == nullptr ? " not" : "");
return JNI_OK;
}

13
src/hotspot/os/windows/os_windows.hpp
View File

@ -109,6 +109,19 @@ class os::win32 {
// load dll from Windows system directory or Windows directory
static HINSTANCE load_Windows_dll(const char* name, char *ebuf, int ebuflen);
// Resolve a symbol from KernelBase.dll, returns nullptr if not found.
static void* lookup_kernelbase_symbol(const char* name);
// VirtualAlloc2 (since Windows version 1803)
// Resolved from KernelBase during os::init_2() or nullptr if unavailable.
typedef PVOID (WINAPI *VirtualAlloc2Fn)(HANDLE, PVOID, SIZE_T, ULONG, ULONG, MEM_EXTENDED_PARAMETER*, ULONG);
static VirtualAlloc2Fn VirtualAlloc2;
// MapViewOfFile3 (since Windows version 1803)
// Resolved from KernelBase during os::init_2() or nullptr if unavailable.
typedef PVOID (WINAPI *MapViewOfFile3Fn)(HANDLE, HANDLE, PVOID, ULONG64, SIZE_T, ULONG, ULONG, MEM_EXTENDED_PARAMETER*, ULONG);
static MapViewOfFile3Fn MapViewOfFile3;
private:
static void initialize_performance_counter();

2
src/hotspot/share/classfile/classFileParser.cpp
View File

@ -154,6 +154,8 @@
#define JAVA_27_VERSION 71
#define JAVA_28_VERSION 72
void ClassFileParser::set_class_bad_constant_seen(short bad_constant) {
assert((bad_constant == JVM_CONSTANT_Module ||
bad_constant == JVM_CONSTANT_Package) && _major_version >= JAVA_9_VERSION,

6
src/hotspot/share/classfile/vmIntrinsics.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2020, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -527,6 +527,10 @@ bool vmIntrinsics::disabled_by_jvm_flags(vmIntrinsics::ID id) {
case vmIntrinsics::_intpoly_assign:
if (!UseIntPolyIntrinsics) return true;
break;
case vmIntrinsics::_intpoly_mult_25519:
case vmIntrinsics::_intpoly_square_25519:
if (!UseIntPoly25519Intrinsics) return true;
break;
case vmIntrinsics::_updateBytesCRC32C:
case vmIntrinsics::_updateDirectByteBufferCRC32C:
if (!UseCRC32CIntrinsics) return true;

7
src/hotspot/share/classfile/vmIntrinsics.hpp
View File

@ -549,6 +549,13 @@ class methodHandle;
do_name(intPolyAssign_name, "conditionalAssign") \
do_signature(intPolyAssign_signature, "(I[J[J)V") \
\
/* support for sun.security.util.math.intpoly.IntegerPolynomial25519 */ \
do_class(sun_security_util_math_intpoly_IntegerPolynomial25519, "sun/security/util/math/intpoly/IntegerPolynomial25519") \
do_intrinsic(_intpoly_mult_25519, sun_security_util_math_intpoly_IntegerPolynomial25519, intPolyMult_name, intPolyMult_signature, F_R) \
do_intrinsic(_intpoly_square_25519, sun_security_util_math_intpoly_IntegerPolynomial25519, intPolySquare_name, intPolySquare_signature, F_R) \
do_name(intPolySquare_name, "square") \
do_signature(intPolySquare_signature, "([J[J)V") \
\
/* support for java.util.Base64.Encoder*/ \
do_class(java_util_Base64_Encoder, "java/util/Base64$Encoder") \
do_intrinsic(_base64_encodeBlock, java_util_Base64_Encoder, encodeBlock_name, encodeBlock_signature, F_R) \

4
src/hotspot/share/gc/g1/g1CollectedHeap.cpp
View File

@ -729,7 +729,7 @@ HeapWord* G1CollectedHeap::attempt_allocation_humongous(size_t word_size) {
result = humongous_obj_allocate(word_size);
if (result != nullptr) {
policy()->old_gen_alloc_tracker()->
add_allocated_humongous_bytes_since_last_gc(humongous_byte_size);
add_allocated_humongous_bytes(humongous_byte_size);
return result;
}
@ -2861,7 +2861,7 @@ void G1CollectedHeap::record_obj_copy_mem_stats() {
G1ReservePercent);
policy()->old_gen_alloc_tracker()->
add_allocated_bytes_since_last_gc(total_old_allocated * HeapWordSize);
add_allocated_non_humongous_bytes(total_old_allocated * HeapWordSize);
_gc_tracer_stw->report_evacuation_statistics(create_g1_evac_summary(&_survivor_evac_stats),
create_g1_evac_summary(&_old_evac_stats));

131
src/hotspot/share/gc/g1/g1ConcurrentCycleTracker.cpp Normal file
View File

@ -0,0 +1,131 @@
/*
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*
*/
#include "gc/g1/g1ConcurrentCycleTracker.hpp"
#include "gc/g1/g1OldGenAllocationTracker.hpp"
#include "utilities/checkedCast.hpp"
#include "utilities/debug.hpp"
void G1ConcurrentCycleTracker::reset() {
_state = CycleState::Inactive;
_total_pause_time_s = 0.0;
_cycle_start_time_s = 0.0;
_cycle_end_time_s = 0.0;
_humongous_bytes_at_start = 0;
_non_humongous_allocated_bytes = 0;
_peak_extra_humongous_occupancy_bytes = 0;
}
void G1ConcurrentCycleTracker::update_allocation_stats(G1AllocationIntervalStats interval_stats) {
if (!is_active()) {
return;
}
_non_humongous_allocated_bytes += interval_stats._non_humongous_allocated_bytes;
intptr_t delta_before = checked_cast<intptr_t>(interval_stats._total_humongous_before_bytes) -
checked_cast<intptr_t>(_humongous_bytes_at_start);
intptr_t delta_after = delta_before +
checked_cast<intptr_t>(interval_stats._humongous_allocated_bytes);
if (delta_after > 0) {
_peak_extra_humongous_occupancy_bytes = MAX2(_peak_extra_humongous_occupancy_bytes, checked_cast<size_t>(delta_after));
}
}
G1ConcurrentCycleTracker::G1ConcurrentCycleTracker()
: _state(CycleState::Inactive),
_cycle_start_time_s(0.0),
_cycle_end_time_s(0.0),
_total_pause_time_s(0.0),
_humongous_bytes_at_start(0),
_non_humongous_allocated_bytes(0),
_peak_extra_humongous_occupancy_bytes(0)
{ }
void G1ConcurrentCycleTracker::record_cycle_start(double cycle_start_time_s, size_t humongous_bytes_after_pause) {
assert(_state == CycleState::Inactive, "Concurrent start out of order.");
_cycle_start_time_s = cycle_start_time_s;
_humongous_bytes_at_start = humongous_bytes_after_pause;
_state = CycleState::Active;
}
void G1ConcurrentCycleTracker::record_allocation_interval(Pause pause_type,
bool is_periodic_gc,
double pause_start_time_s,
double pause_end_time_s,
G1AllocationIntervalStats interval_stats) {
if (is_periodic_gc || pause_type == Pause::Full || pause_type == Pause::ConcurrentStartUndo) {
reset();
return;
}
if (pause_type == Pause::ConcurrentStartFull) {
record_cycle_start(pause_end_time_s, interval_stats._total_humongous_after_bytes);
return;
}
if (!is_active()) {
return;
}
update_allocation_stats(interval_stats);
if (pause_type == Pause::Mixed) {
complete_cycle(pause_start_time_s);
return;
}
assert(pause_type == Pause::Normal ||
pause_type == Pause::PrepareMixed ||
pause_type == Pause::Remark ||
pause_type == Pause::Cleanup,
"Unhandled pause type");
add_pause(pause_end_time_s - pause_start_time_s);
}
void G1ConcurrentCycleTracker::complete_cycle(double cycle_end_time_s) {
precond(is_active());
_cycle_end_time_s = cycle_end_time_s;
_state = CycleState::Complete;
}
G1ConcurrentCycleStats G1ConcurrentCycleTracker::get_and_reset_cycle_stats() {
precond(has_completed_cycle());
double cycle_duration = (_cycle_end_time_s - _cycle_start_time_s - _total_pause_time_s);
G1ConcurrentCycleStats stats{
cycle_duration,
_non_humongous_allocated_bytes,
_peak_extra_humongous_occupancy_bytes
};
reset();
return stats;
}

111
src/hotspot/share/gc/g1/g1ConcurrentCycleTracker.hpp Normal file
View File

@ -0,0 +1,111 @@
/*
* Copyright (c) 2001, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*
*/
#ifndef SHARE_GC_G1_G1CONCURRENTCYCLETRACKER_HPP
#define SHARE_GC_G1_G1CONCURRENTCYCLETRACKER_HPP
#include "gc/g1/g1CollectorState.hpp"
#include "utilities/globalDefinitions.hpp"
struct G1AllocationIntervalStats;
// The sampling interval for G1ConcurrentCycleTracker covers the concurrent cycle
// from the end of the Concurrent Start GC to start of the first Mixed GC.
struct G1ConcurrentCycleStats {
double _cycle_duration_s;
size_t _non_humongous_allocated_bytes;
size_t _peak_extra_humongous_occupancy_bytes;
G1ConcurrentCycleStats(double cycle_duration_s,
size_t non_humongous_allocated_bytes,
size_t peak_extra_humongous_occupancy_bytes)
: _cycle_duration_s(cycle_duration_s),
_non_humongous_allocated_bytes(non_humongous_allocated_bytes),
_peak_extra_humongous_occupancy_bytes(peak_extra_humongous_occupancy_bytes)
{ }
};
class G1ConcurrentCycleTracker {
using Pause = G1CollectorState::Pause;
enum class CycleState {
Inactive,
Active,
Complete,
};
CycleState _state;
double _cycle_start_time_s;
double _cycle_end_time_s;
double _total_pause_time_s;
// allocation accounting
size_t _humongous_bytes_at_start;
size_t _non_humongous_allocated_bytes;
size_t _peak_extra_humongous_occupancy_bytes;
void reset();
bool is_active() const {
return _state == CycleState::Active;
}
void update_allocation_stats(G1AllocationIntervalStats interval_stats);
void add_pause(double pause_duration_s) {
_total_pause_time_s += pause_duration_s;
}
void record_cycle_start(double cycle_start_time_s, size_t humongous_bytes_after_pause);
void complete_cycle(double cycle_end_time_s);
public:
G1ConcurrentCycleTracker();
void record_allocation_interval(Pause pause_type,
bool is_periodic_gc,
double pause_start_time_s,
double pause_end_time_s,
G1AllocationIntervalStats interval_stats);
void abort_cycle() {
reset();
}
bool has_completed_cycle() const {
return _state == CycleState::Complete;
}
size_t non_humongous_allocated_bytes() const {
return _non_humongous_allocated_bytes;
}
size_t peak_extra_humongous_occupancy_bytes() const {
return _peak_extra_humongous_occupancy_bytes;
}
G1ConcurrentCycleStats get_and_reset_cycle_stats();
};
#endif // SHARE_GC_G1_G1CONCURRENTCYCLETRACKER_HPP

15
src/hotspot/share/gc/g1/g1ConcurrentMark.cpp
View File

@ -714,6 +714,11 @@ private:
}
HeapWord* region_clear_limit(G1HeapRegion* r) {
// A garbage collection might have made the region unavailable after a yield during
// clearing. Just return bottom as the limit, causing the clearing for this region to end.
if (G1CollectedHeap::heap()->region_at_or_null(r->hrm_index()) == nullptr) {
return r->bottom();
}
// During a Concurrent Undo Mark cycle, the per region top_at_mark_start and
// live_words data are current wrt to the _mark_bitmap. We use this information
// to only clear ranges of the bitmap that require clearing.
@ -743,7 +748,7 @@ private:
}
HeapWord* cur = r->bottom();
HeapWord* const end = region_clear_limit(r);
HeapWord* end = region_clear_limit(r);
size_t const chunk_size_in_words = G1ClearBitMapTask::chunk_size() / HeapWordSize;
@ -761,8 +766,12 @@ private:
assert(!suspendible() || _cm->is_in_reset_for_next_cycle(), "invariant");
// Abort iteration if necessary.
if (has_aborted()) {
return true;
if (suspendible() && _cm->do_yield_check()) {
if (_cm->has_aborted()) {
return true;
}
// Re-read end. The region might have been uncommitted.
end = region_clear_limit(r);
}
}
assert(cur >= end, "Must have completed iteration over the bitmap for region %u.", r->hrm_index());

89
src/hotspot/share/gc/g1/g1ConcurrentStartToMixedTimeTracker.hpp
View File

@ -1,89 +0,0 @@
/*
* Copyright (c) 2001, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*
*/
#ifndef SHARE_GC_G1_G1CONCURRENTSTARTTOMIXEDTIMETRACKER_HPP
#define SHARE_GC_G1_G1CONCURRENTSTARTTOMIXEDTIMETRACKER_HPP
#include "utilities/debug.hpp"
#include "utilities/globalDefinitions.hpp"
// Used to track time from the end of concurrent start to the first mixed GC.
// After calling the concurrent start/mixed gc notifications, the result can be
// obtained in get_and_reset_last_marking_time() once, after which the tracking resets.
// Any pauses recorded by add_pause() will be subtracted from that results.
class G1ConcurrentStartToMixedTimeTracker {
private:
bool _active;
double _concurrent_start_end_time;
double _mixed_start_time;
double _total_pause_time;
double wall_time() const {
return _mixed_start_time - _concurrent_start_end_time;
}
public:
G1ConcurrentStartToMixedTimeTracker() { reset(); }
// Record concurrent start pause end, starting the time tracking.
void record_concurrent_start_end(double end_time) {
assert(!_active, "Concurrent start out of order.");
_concurrent_start_end_time = end_time;
_active = true;
}
// Record the first mixed gc pause start, ending the time tracking.
void record_mixed_gc_start(double start_time) {
if (_active) {
_mixed_start_time = start_time;
_active = false;
}
}
double get_and_reset_last_marking_time() {
assert(has_result(), "Do not have all measurements yet.");
double result = (_mixed_start_time - _concurrent_start_end_time) - _total_pause_time;
reset();
return result;
}
void reset() {
_active = false;
_total_pause_time = 0.0;
_concurrent_start_end_time = -1.0;
_mixed_start_time = -1.0;
}
void add_pause(double time) {
if (_active) {
_total_pause_time += time;
}
}
bool is_active() const { return _active; }
// Returns whether we have a result that can be retrieved.
bool has_result() const { return _mixed_start_time > 0.0 && _concurrent_start_end_time > 0.0; }
};
#endif // SHARE_GC_G1_G1CONCURRENTSTARTTOMIXEDTIMETRACKER_HPP

136
src/hotspot/share/gc/g1/g1IHOPControl.cpp
View File

@ -31,19 +31,14 @@
double G1IHOPControl::predict(const TruncatedSeq* seq) const {
assert(_is_adaptive, "precondition");
assert(_predictor != nullptr, "precondition");
return _predictor->predict_zero_bounded(seq);
return _predictor->predict_zero_bounded(seq);
}
bool G1IHOPControl::have_enough_data_for_prediction() const {
assert(_is_adaptive, "precondition");
return ((size_t)_marking_start_to_mixed_time_s.num() >= G1AdaptiveIHOPNumInitialSamples) &&
((size_t)_old_gen_alloc_rate.num() >= G1AdaptiveIHOPNumInitialSamples);
}
double G1IHOPControl::last_marking_start_to_mixed_time_s() const {
return _marking_start_to_mixed_time_s.last();
((size_t)_old_non_humongous_alloc_rate.num() >= G1AdaptiveIHOPNumInitialSamples);
}
size_t G1IHOPControl::effective_target_occupancy() const {
@ -66,7 +61,6 @@ size_t G1IHOPControl::effective_target_occupancy() const {
}
G1IHOPControl::G1IHOPControl(double ihop_percent,
const G1OldGenAllocationTracker* old_gen_alloc_tracker,
bool adaptive,
const G1Predictions* predictor,
size_t heap_reserve_percent,
@ -76,11 +70,10 @@ G1IHOPControl::G1IHOPControl(double ihop_percent,
_target_occupancy(0),
_heap_reserve_percent(heap_reserve_percent),
_heap_waste_percent(heap_waste_percent),
_last_allocation_time_s(0.0),
_old_gen_alloc_tracker(old_gen_alloc_tracker),
_predictor(predictor),
_marking_start_to_mixed_time_s(10, 0.05),
_old_gen_alloc_rate(10, 0.05),
_old_non_humongous_alloc_rate(10, 0.05),
_peak_extra_humongous_occupancy_in_mark_cycle(10, 0.05),
_expected_young_gen_at_first_mixed_gc(0) {
assert(_initial_ihop_percent >= 0.0 && _initial_ihop_percent <= 100.0,
"IHOP percent out of range: %.3f", ihop_percent);
@ -93,22 +86,28 @@ void G1IHOPControl::update_target_occupancy(size_t new_target_occupancy) {
_target_occupancy = new_target_occupancy;
}
void G1IHOPControl::report_statistics(G1NewTracer* new_tracer, size_t non_young_occupancy) {
print_log(non_young_occupancy);
send_trace_event(new_tracer, non_young_occupancy);
void G1IHOPControl::report_statistics(G1NewTracer* new_tracer,
size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy) {
print_log(non_young_occupancy, non_humongous_allocation, peak_extra_humongous_occupancy);
send_trace_event(new_tracer, non_young_occupancy,
non_humongous_allocation, peak_extra_humongous_occupancy);
}
void G1IHOPControl::update_allocation_info(double allocation_time_s, size_t expected_young_gen_size) {
assert(allocation_time_s > 0, "Invalid allocation time: %.3f", allocation_time_s);
_last_allocation_time_s = allocation_time_s;
double alloc_rate = _old_gen_alloc_tracker->last_period_old_gen_growth() / allocation_time_s;
_old_gen_alloc_rate.add(alloc_rate);
void G1IHOPControl::record_expected_young_gen_size(size_t expected_young_gen_size) {
_expected_young_gen_at_first_mixed_gc = expected_young_gen_size;
}
void G1IHOPControl::add_marking_start_to_mixed_length(double length_s) {
assert(length_s >= 0.0, "Invalid marking length: %.3f", length_s);
_marking_start_to_mixed_time_s.add(length_s);
void G1IHOPControl::record_concurrent_cycle(double marking_start_to_mixed_time_s,
size_t non_humongous_bytes,
size_t peak_extra_humongous_occupancy_bytes) {
assert(marking_start_to_mixed_time_s > 0.0, "Invalid concurrent cycle duration: %.3f", marking_start_to_mixed_time_s);
double non_humongous_rate = non_humongous_bytes / marking_start_to_mixed_time_s;
_marking_start_to_mixed_time_s.add(marking_start_to_mixed_time_s);
_old_non_humongous_alloc_rate.add(non_humongous_rate);
_peak_extra_humongous_occupancy_in_mark_cycle.add(peak_extra_humongous_occupancy_bytes);
}
// Determine the old generation occupancy threshold at which to start
@ -121,80 +120,93 @@ size_t G1IHOPControl::old_gen_threshold_for_conc_mark_start() const {
return (size_t)(_initial_ihop_percent * _target_occupancy / 100.0);
}
// During the time between marking start and the first Mixed GC,
// additional memory will be consumed:
// - Old gen grows due to allocations:
// old_gen_alloc_bytes = old_gen_alloc_rate * marking_start_to_mixed_time
// - Young gen will occupy a certain size at the first Mixed GC:
// expected_young_gen_at_first_mixed_gc
double marking_start_to_mixed_time = predict(&_marking_start_to_mixed_time_s);
double old_gen_alloc_rate = predict(&_old_gen_alloc_rate);
size_t old_gen_alloc_bytes = (size_t)(marking_start_to_mixed_time * old_gen_alloc_rate);
// Therefore, the total heap occupancy at the first Mixed GC is:
// current_old_gen + old_gen_growth + expected_young_gen_at_first_mixed_gc
// Between Concurrent Start GC and the first Mixed GC (i.e. concurrent cycle),
// we expect extra heap occupancy from three sources:
// - non-humongous allocations into the old-gen
// - peak extra humongous occupancy during the cycle, relative to the humongous occupancy
// at the end of the Concurrent Start GC.
// - we also wish to maintain the current desired young generation until the first Mixed-gc;
// promotions into the old gen should not shrink the young gen and degrade performance.
//
// To ensure this does not exceed the target_heap_occupancy, we work
// backwards to compute the old gen occupancy at which marking must start:
// mark_start_threshold = target_heap_occupancy -
// (old_gen_growth + expected_young_gen_at_first_mixed_gc)
// We therefore start marking early enough such that:
//
// old_gen_at_concurrent_start +
// predicted_non_hum_old_growth +
// predicted_peak_extra_humongous_occupancy +
// expected_young_gen_at_first_mixed_gc
//
// stays below the effective target occupancy.
double marking_start_to_mixed_time = predict(&_marking_start_to_mixed_time_s);
double old_non_humongous_alloc_rate = predict(&_old_non_humongous_alloc_rate);
size_t old_non_humongous_alloc_bytes = (size_t)(marking_start_to_mixed_time * old_non_humongous_alloc_rate);
size_t predicted_needed = old_gen_alloc_bytes + _expected_young_gen_at_first_mixed_gc;
size_t predicted_peak_extra_humongous_occupancy =
predict(&_peak_extra_humongous_occupancy_in_mark_cycle);
size_t reserve_for_young_regions = _expected_young_gen_at_first_mixed_gc;
size_t target_heap_occupancy = effective_target_occupancy();
return predicted_needed < target_heap_occupancy
? target_heap_occupancy - predicted_needed
: 0;
size_t needed_for_concurrent_cycle = reserve_for_young_regions +
old_non_humongous_alloc_bytes +
predicted_peak_extra_humongous_occupancy;
size_t threshold = needed_for_concurrent_cycle < target_heap_occupancy ?
target_heap_occupancy - needed_for_concurrent_cycle : 0;
return threshold;
}
void G1IHOPControl::print_log(size_t non_young_occupancy) {
void G1IHOPControl::print_log(size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy) {
assert(_target_occupancy > 0, "Target occupancy still not updated yet.");
size_t old_gen_mark_start_threshold = old_gen_threshold_for_conc_mark_start();
log_debug(gc, ihop)("Basic information (value update), old-gen threshold: %zuB (%1.2f%%), target occupancy: %zuB, old-gen occupancy: %zuB (%1.2f%%), "
"recent old-gen allocation size: %zuB, recent allocation duration: %1.2fms, recent old-gen allocation rate: %1.2fB/s, recent marking phase length: %1.2fms",
log_debug(gc, ihop)("Basic information (value update), old-gen threshold: %zuB (%1.2f%%), target occupancy: %zuB, old-gen occupancy: %zuB (%1.2f%%)",
old_gen_mark_start_threshold,
percent_of(old_gen_mark_start_threshold, _target_occupancy),
_target_occupancy,
non_young_occupancy,
percent_of(non_young_occupancy, _target_occupancy),
_old_gen_alloc_tracker->last_period_old_gen_bytes(),
_last_allocation_time_s * 1000.0,
_last_allocation_time_s > 0.0 ? _old_gen_alloc_tracker->last_period_old_gen_bytes() / _last_allocation_time_s : 0.0,
last_marking_start_to_mixed_time_s() * 1000.0);
percent_of(non_young_occupancy, _target_occupancy));
if (!_is_adaptive) {
if (!_is_adaptive || !have_enough_data_for_prediction()) {
return;
}
size_t effective_target = effective_target_occupancy();
log_debug(gc, ihop)("Adaptive IHOP information (value update), prediction active: %s, old-gen threshold: %zuB (%1.2f%%), internal target occupancy: %zuB, "
"old-gen occupancy: %zuB, additional buffer size: %zuB, predicted old-gen allocation rate: %1.2fB/s, "
"predicted marking phase length: %1.2fms",
BOOL_TO_STR(have_enough_data_for_prediction()),
log_debug(gc, ihop)("Adaptive IHOP information (value update), old-gen threshold: %zuB (%1.2f%%), internal target occupancy: %zuB, "
"old-gen occupancy: %zuB (%1.2f%%), additional buffer size: %zuB, "
"current non-humongous allocation: %zuB, current peak extra humongous occupancy: %zuB, "
"predicted old-gen non-humongous allocation rate: %1.2fB/s, predicted peak extra humongous occupancy: %1.2fB, "
"predicted concurrent cycle duration: %1.2fms",
old_gen_mark_start_threshold,
percent_of(old_gen_mark_start_threshold, effective_target),
effective_target,
non_young_occupancy,
percent_of(non_young_occupancy, effective_target),
_expected_young_gen_at_first_mixed_gc,
predict(&_old_gen_alloc_rate),
non_humongous_allocation, peak_extra_humongous_occupancy,
predict(&_old_non_humongous_alloc_rate),
predict(&_peak_extra_humongous_occupancy_in_mark_cycle),
predict(&_marking_start_to_mixed_time_s) * 1000.0);
}
void G1IHOPControl::send_trace_event(G1NewTracer* tracer, size_t non_young_occupancy) {
void G1IHOPControl::send_trace_event(G1NewTracer* tracer,
size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy) {
assert(_target_occupancy > 0, "Target occupancy still not updated yet.");
tracer->report_basic_ihop_statistics(old_gen_threshold_for_conc_mark_start(),
_target_occupancy,
non_young_occupancy,
_old_gen_alloc_tracker->last_period_old_gen_bytes(),
_last_allocation_time_s,
last_marking_start_to_mixed_time_s());
non_young_occupancy);
if (_is_adaptive) {
tracer->report_adaptive_ihop_statistics(old_gen_threshold_for_conc_mark_start(),
effective_target_occupancy(),
non_young_occupancy,
_expected_young_gen_at_first_mixed_gc,
predict(&_old_gen_alloc_rate),
non_humongous_allocation,
peak_extra_humongous_occupancy,
predict(&_old_non_humongous_alloc_rate),
predict(&_peak_extra_humongous_occupancy_in_mark_cycle),
predict(&_marking_start_to_mixed_time_s),
have_enough_data_for_prediction());
}

45
src/hotspot/share/gc/g1/g1IHOPControl.hpp
View File

@ -25,7 +25,6 @@
#ifndef SHARE_GC_G1_G1IHOPCONTROL_HPP
#define SHARE_GC_G1_G1IHOPCONTROL_HPP
#include "gc/g1/g1OldGenAllocationTracker.hpp"
#include "memory/allocation.hpp"
#include "utilities/numberSeq.hpp"
@ -53,16 +52,15 @@ class G1IHOPControl : public CHeapObj<mtGC> {
// Percentage of free heap that should be considered as waste.
const size_t _heap_waste_percent;
// Most recent complete mutator allocation period in seconds.
double _last_allocation_time_s;
const G1OldGenAllocationTracker* _old_gen_alloc_tracker;
const G1Predictions* _predictor;
// Wall-clock time in seconds from marking start to the first mixed GC,
// excluding GC Pause time.
TruncatedSeq _marking_start_to_mixed_time_s;
// Old generation allocation rate in bytes per second.
TruncatedSeq _old_gen_alloc_rate;
// Track old-generation allocations during a concurrent cycle: end of the
// Concurrent Start to the first Mixed GC.
// These values are used only when G1UseAdaptiveIHOP is enabled.
TruncatedSeq _old_non_humongous_alloc_rate;
TruncatedSeq _peak_extra_humongous_occupancy_in_mark_cycle;
// The most recent unrestrained size of the young gen. This is used as an additional
// factor in the calculation of the threshold, as the threshold is based on
@ -77,19 +75,23 @@ class G1IHOPControl : public CHeapObj<mtGC> {
double predict(const TruncatedSeq* seq) const;
bool have_enough_data_for_prediction() const;
double last_marking_start_to_mixed_time_s() const;
// The "effective" target occupancy the algorithm wants to keep until the start
// of Mixed GCs. This is typically lower than the target occupancy, as the
// algorithm needs to consider restrictions by the environment.
size_t effective_target_occupancy() const;
void print_log(size_t non_young_occupancy);
void send_trace_event(G1NewTracer* tracer, size_t non_young_occupancy);
void print_log(size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy);
void send_trace_event(G1NewTracer* tracer,
size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy);
public:
G1IHOPControl(double ihop_percent,
const G1OldGenAllocationTracker* old_gen_alloc_tracker,
bool adaptive,
const G1Predictions* predictor,
size_t heap_reserve_percent,
@ -98,26 +100,23 @@ class G1IHOPControl : public CHeapObj<mtGC> {
// Adjust target occupancy.
void update_target_occupancy(size_t new_target_occupancy);
void update_target_after_marking_phase();
// Update allocation rate information and current expected young gen size for the
// first mixed gc needed for the predictor. Allocation rate is given as the
// separately passed in allocation increment and the time passed (mutator time)
// for the latest allocation increment here. Allocation size is the memory needed
// during the mutator before and the first mixed gc pause itself.
// Updates expected young gen size for the first mixed gc needed for the predictor.
// Contents include young gen at that point, and the memory required for evacuating
// the collection set in that first mixed gc (including waste caused by PLAB
// allocation etc.).
void update_allocation_info(double allocation_time_s, size_t expected_young_gen_size);
void record_expected_young_gen_size(size_t expected_young_gen_size);
// Update the time spent in the mutator beginning from the end of concurrent start to
// the first mixed gc.
void add_marking_start_to_mixed_length(double length_s);
void record_concurrent_cycle(double marking_start_to_mixed_time_s,
size_t non_humongous_bytes,
size_t peak_extra_humongous_occupancy);
// Get the current non-young occupancy at which concurrent marking should start.
size_t old_gen_threshold_for_conc_mark_start() const;
void report_statistics(G1NewTracer* tracer, size_t non_young_occupancy);
void report_statistics(G1NewTracer* tracer,
size_t non_young_occupancy,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy);
};
#endif // SHARE_GC_G1_G1IHOPCONTROL_HPP

51
src/hotspot/share/gc/g1/g1OldGenAllocationTracker.cpp
View File

@ -26,36 +26,43 @@
#include "logging/log.hpp"
G1OldGenAllocationTracker::G1OldGenAllocationTracker() :
_last_period_old_gen_bytes(0),
_last_period_old_gen_growth(0),
_humongous_bytes_after_last_gc(0),
_allocated_bytes_since_last_gc(0),
_allocated_humongous_bytes_since_last_gc(0) {
_humongous_bytes_after_last_pause(0),
_allocated_non_humongous_bytes_since_last_pause(0),
_allocated_humongous_bytes_since_last_pause(0) {
}
void G1OldGenAllocationTracker::reset_after_gc(size_t humongous_bytes_after_gc) {
G1AllocationIntervalStats G1OldGenAllocationTracker::end_allocation_interval(size_t humongous_bytes_after_pause) {
// Calculate actual increase in old, taking eager reclaim into consideration.
size_t last_period_humongous_increase = 0;
if (humongous_bytes_after_gc > _humongous_bytes_after_last_gc) {
last_period_humongous_increase = humongous_bytes_after_gc - _humongous_bytes_after_last_gc;
assert(last_period_humongous_increase <= _allocated_humongous_bytes_since_last_gc,
size_t last_interval_humongous_increase = 0;
if (humongous_bytes_after_pause > _humongous_bytes_after_last_pause) {
last_interval_humongous_increase = humongous_bytes_after_pause - _humongous_bytes_after_last_pause;
assert(last_interval_humongous_increase <= _allocated_humongous_bytes_since_last_pause,
"Increase larger than allocated %zu <= %zu",
last_period_humongous_increase, _allocated_humongous_bytes_since_last_gc);
last_interval_humongous_increase, _allocated_humongous_bytes_since_last_pause);
}
_last_period_old_gen_growth = _allocated_bytes_since_last_gc + last_period_humongous_increase;
// Calculate and record needed values.
_last_period_old_gen_bytes = _allocated_bytes_since_last_gc + _allocated_humongous_bytes_since_last_gc;
_humongous_bytes_after_last_gc = humongous_bytes_after_gc;
size_t last_interval_old_gen_growth = _allocated_non_humongous_bytes_since_last_pause +
last_interval_humongous_increase;
log_debug(gc, alloc, stats)("Old generation allocation in the last mutator period, "
G1AllocationIntervalStats allocation_interval_stats{
_allocated_non_humongous_bytes_since_last_pause,
_allocated_humongous_bytes_since_last_pause,
_humongous_bytes_after_last_pause,
humongous_bytes_after_pause
};
_humongous_bytes_after_last_pause = humongous_bytes_after_pause;
log_debug(gc, alloc, stats)("Old generation allocation in the last allocation interval, "
"old gen allocated: %zuB, humongous allocated: %zuB, "
"old gen growth: %zuB.",
_allocated_bytes_since_last_gc,
_allocated_humongous_bytes_since_last_gc,
_last_period_old_gen_growth);
_allocated_non_humongous_bytes_since_last_pause,
_allocated_humongous_bytes_since_last_pause,
last_interval_old_gen_growth);
// Reset for next mutator period.
_allocated_bytes_since_last_gc = 0;
_allocated_humongous_bytes_since_last_gc = 0;
// Reset for the next interval.
_allocated_non_humongous_bytes_since_last_pause = 0;
_allocated_humongous_bytes_since_last_pause = 0;
return allocation_interval_stats;
}

81
src/hotspot/share/gc/g1/g1OldGenAllocationTracker.hpp
View File

@ -22,46 +22,71 @@
*
*/
#ifndef SHARE_VM_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP
#define SHARE_VM_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP
#ifndef SHARE_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP
#define SHARE_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP
#include "gc/g1/g1HeapRegion.hpp"
#include "memory/allocation.hpp"
// Allocation statistics for an allocation interval, i.e. the interval between two
// consecutive STW pauses.
//
// The allocation counters record allocations made during that interval.
//
// _total_humongous_before_bytes is the humongous occupancy after the previous
// pause (at the start of the allocation interval).
//
// _total_humongous_after_bytes is the humongous occupancy after the current
// pause (at the end of the allocation interval).
struct G1AllocationIntervalStats {
size_t _non_humongous_allocated_bytes;
size_t _humongous_allocated_bytes;
size_t _total_humongous_before_bytes;
size_t _total_humongous_after_bytes;
G1AllocationIntervalStats(size_t non_humongous_allocated_bytes,
size_t humongous_allocated_bytes,
size_t total_humongous_before_bytes,
size_t total_humongous_after_bytes)
: _non_humongous_allocated_bytes(non_humongous_allocated_bytes),
_humongous_allocated_bytes(humongous_allocated_bytes),
_total_humongous_before_bytes(total_humongous_before_bytes),
_total_humongous_after_bytes(total_humongous_after_bytes)
{ }
void record_humongous_allocation(size_t humongous_allocation_bytes) {
_humongous_allocated_bytes += humongous_allocation_bytes;
_total_humongous_after_bytes += humongous_allocation_bytes;
}
};
// Track allocation details in the old generation.
class G1OldGenAllocationTracker : public CHeapObj<mtGC> {
// Total number of bytes allocated in the old generation at the end
// of the last gc.
size_t _last_period_old_gen_bytes;
// Total growth of the old geneneration since the last gc,
// taking eager-reclaim into consideration.
size_t _last_period_old_gen_growth;
// Total size of humongous objects after the last STW pause.
size_t _humongous_bytes_after_last_pause;
// Total size of humongous objects for last gc.
size_t _humongous_bytes_after_last_gc;
// Non-humongous old generation allocations since the last gc.
size_t _allocated_bytes_since_last_gc;
// Humongous allocations during last mutator period.
size_t _allocated_humongous_bytes_since_last_gc;
// Non-humongous old generation allocations since the last STW pause.
size_t _allocated_non_humongous_bytes_since_last_pause;
// Humongous allocations during last allocation interval.
size_t _allocated_humongous_bytes_since_last_pause;
public:
G1OldGenAllocationTracker();
void add_allocated_bytes_since_last_gc(size_t bytes) { _allocated_bytes_since_last_gc += bytes; }
void add_allocated_humongous_bytes_since_last_gc(size_t bytes) { _allocated_humongous_bytes_since_last_gc += bytes; }
// Record a humongous allocation in a collection pause. This allocation
// is accounted to the previous mutator period.
void record_collection_pause_humongous_allocation(size_t bytes) {
_humongous_bytes_after_last_gc += bytes;
void add_allocated_non_humongous_bytes(size_t bytes) {
_allocated_non_humongous_bytes_since_last_pause += bytes;
}
void add_allocated_humongous_bytes(size_t bytes) {
_allocated_humongous_bytes_since_last_pause += bytes;
}
size_t last_period_old_gen_bytes() const { return _last_period_old_gen_bytes; }
size_t last_period_old_gen_growth() const { return _last_period_old_gen_growth; };
// Record a humongous allocation in a collection pause. This allocation
// is accounted to the previous allocation interval.
void record_collection_pause_humongous_allocation(size_t bytes) {
_humongous_bytes_after_last_pause += bytes;
}
// Calculates and resets stats after a collection.
void reset_after_gc(size_t humongous_bytes_after_gc);
// Calculate and reset allocation statistics after a pause.
G1AllocationIntervalStats end_allocation_interval(size_t humongous_bytes_after_pause);
};
#endif // SHARE_VM_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP
#endif // SHARE_GC_G1_G1OLDGENALLOCATIONTRACKER_HPP

172
src/hotspot/share/gc/g1/g1Policy.cpp
View File

@ -55,8 +55,9 @@ G1Policy::G1Policy(STWGCTimer* gc_timer) :
_analytics(new G1Analytics(&_predictor)),
_remset_tracker(),
_mmu_tracker(new G1MMUTracker(GCPauseIntervalMillis / 1000.0, MaxGCPauseMillis / 1000.0)),
_concurrent_cycle_tracker(),
_old_gen_alloc_tracker(),
_ihop_control(create_ihop_control(&_old_gen_alloc_tracker, &_predictor)),
_ihop_control(create_ihop_control(&_predictor)),
_policy_counters(new GCPolicyCounters("GarbageFirst", 1, 2)),
_cur_pause_start_sec(0.0),
_young_list_desired_length(0),
@ -68,7 +69,6 @@ G1Policy::G1Policy(STWGCTimer* gc_timer) :
_young_gen_sizer(),
_free_regions_at_end_of_collection(0),
_pending_cards_from_gc(0),
_concurrent_start_to_mixed(),
_collection_set(nullptr),
_g1h(nullptr),
_phase_times_timer(gc_timer),
@ -160,7 +160,7 @@ void G1Policy::record_new_heap_size(uint new_number_of_regions) {
double reserve_regions_d = (double) new_number_of_regions * _reserve_factor;
// We use ceiling so that if reserve_regions_d is > 0.0 (but
// smaller than 1.0) we'll get 1.
_reserve_regions = (uint) ceil(reserve_regions_d);
_reserve_regions.store_relaxed((uint) ceil(reserve_regions_d));
_young_gen_sizer.heap_size_changed(new_number_of_regions);
@ -186,8 +186,22 @@ void G1Policy::update_young_length_bounds() {
void G1Policy::update_young_length_bounds(size_t pending_cards, size_t card_rs_length, size_t code_root_rs_length) {
uint old_young_list_target_length = young_list_target_length();
uint new_young_list_desired_length = calculate_young_desired_length(pending_cards, card_rs_length, code_root_rs_length);
uint new_young_list_target_length = calculate_young_target_length(new_young_list_desired_length);
uint min_young_length_by_sizer = _young_gen_sizer.min_desired_young_length();
uint max_young_length_by_sizer = _young_gen_sizer.max_desired_young_length();
if (max_young_length_by_sizer < min_young_length_by_sizer) {
// This can happen due to races with heap_size_changed() at mutator time. Do not update the young gen
// lengths. Will be updated on the next regular call anyway.
assert(!SafepointSynchronize::is_at_safepoint(), "must be");
return;
}
uint new_young_list_desired_length = calculate_young_desired_length(pending_cards,
card_rs_length,
code_root_rs_length,
min_young_length_by_sizer,
max_young_length_by_sizer);
uint new_young_list_target_length = calculate_young_target_length(new_young_list_desired_length, min_young_length_by_sizer);
log_trace(gc, ergo, heap)("Young list length update: pending cards %zu card_rs_length %zu old target %u desired: %u target: %u",
pending_cards,
@ -224,9 +238,9 @@ void G1Policy::update_young_length_bounds(size_t pending_cards, size_t card_rs_l
//
uint G1Policy::calculate_young_desired_length(size_t pending_cards,
size_t card_rs_length,
size_t code_root_rs_length) const {
uint min_young_length_by_sizer = _young_gen_sizer.min_desired_young_length();
uint max_young_length_by_sizer = _young_gen_sizer.max_desired_young_length();
size_t code_root_rs_length,
uint min_young_length_by_sizer,
uint max_young_length_by_sizer) const {
assert(min_young_length_by_sizer >= 1, "invariant");
assert(max_young_length_by_sizer >= min_young_length_by_sizer, "invariant");
@ -302,7 +316,7 @@ uint G1Policy::calculate_young_desired_length(size_t pending_cards,
// Limit the desired (wished) young length by current free regions. If the request
// can be satisfied without using up reserve regions, do so, otherwise eat into
// the reserve, giving away at most what the heap sizer allows.
uint G1Policy::calculate_young_target_length(uint desired_young_length) const {
uint G1Policy::calculate_young_target_length(uint desired_young_length, uint min_young_length_by_sizer) const {
uint allocated_young_length = _g1h->young_regions_count();
uint receiving_additional_eden;
@ -319,8 +333,14 @@ uint G1Policy::calculate_young_target_length(uint desired_young_length) const {
// do, we at most eat the sizer's minimum regions into the reserve or half the
// reserve rounded up (if possible; this is an arbitrary value).
uint max_to_eat_into_reserve = MIN2(_young_gen_sizer.min_desired_young_length(),
(_reserve_regions + 1) / 2);
// The heap reserve needs to be snapshotted for consistent use in the following.
// It can be concurrently modified by the mutator as it expands the heap. It can
// only increase at that time, so this is a conservative snapshot. So at worst this
// method will return a too small young gen length in that case.
uint reserve_regions = _reserve_regions.load_relaxed();
uint max_to_eat_into_reserve = MIN2(min_young_length_by_sizer,
(reserve_regions + 1) / 2);
log_trace(gc, ergo, heap)("Young target length: Common "
"free regions at end of collection %u "
@ -329,14 +349,14 @@ uint G1Policy::calculate_young_target_length(uint desired_young_length) const {
"max to eat into reserve %u",
_free_regions_at_end_of_collection,
desired_young_length,
_reserve_regions,
reserve_regions,
max_to_eat_into_reserve);
uint survivor_regions_count = _g1h->survivor_regions_count();
uint desired_eden_length = desired_young_length - survivor_regions_count;
uint allocated_eden_length = allocated_young_length - survivor_regions_count;
if (_free_regions_at_end_of_collection <= _reserve_regions) {
if (_free_regions_at_end_of_collection <= reserve_regions) {
// Fully eat (or already eating) into the reserve, hand back at most absolute_min_length regions.
uint receiving_eden = MIN3(_free_regions_at_end_of_collection,
desired_eden_length,
@ -351,9 +371,9 @@ uint G1Policy::calculate_young_target_length(uint desired_young_length) const {
log_trace(gc, ergo, heap)("Young target length: Fully eat into reserve "
"receiving eden %u receiving additional eden %u",
receiving_eden, receiving_additional_eden);
} else if (_free_regions_at_end_of_collection < (desired_eden_length + _reserve_regions)) {
} else if (_free_regions_at_end_of_collection < (desired_eden_length + reserve_regions)) {
// Partially eat into the reserve, at most max_to_eat_into_reserve regions.
uint free_outside_reserve = _free_regions_at_end_of_collection - _reserve_regions;
uint free_outside_reserve = _free_regions_at_end_of_collection - reserve_regions;
assert(free_outside_reserve < desired_eden_length,
"must be %u %u",
free_outside_reserve, desired_eden_length);
@ -569,6 +589,7 @@ void G1Policy::record_full_collection_end(size_t allocation_word_size) {
// Consider this like a collection pause for the purposes of allocation
// since last pause.
double end_sec = os::elapsedTime();
double start_time_sec = cur_pause_start_sec();
// "Nuke" the heuristics that control the young/mixed GC
// transitions and make sure we start with young GCs after the Full GC.
@ -582,9 +603,6 @@ void G1Policy::record_full_collection_end(size_t allocation_word_size) {
_survivor_surv_rate_group->reset();
update_young_length_bounds();
_old_gen_alloc_tracker.reset_after_gc(_g1h->humongous_regions_count() * G1HeapRegion::GrainBytes);
double start_time_sec = cur_pause_start_sec();
record_pause(Pause::Full, start_time_sec, end_sec);
}
@ -934,8 +952,6 @@ G1CollectorState G1Policy::record_young_collection_end(bool concurrent_operation
phase_times()->sum_thread_work_items(G1GCPhaseTimes::MergePSS, G1GCPhaseTimes::MergePSSToYoungGenCards));
}
record_pause(collector_state()->gc_pause_type(concurrent_operation_is_full_mark), start_time_sec, end_time_sec);
if (collector_state()->is_in_prepare_mixed_gc()) {
assert(!collector_state()->is_in_concurrent_start_gc(),
"The young GC before mixed is not allowed to be concurrent start GC");
@ -968,23 +984,27 @@ G1CollectorState G1Policy::record_young_collection_end(bool concurrent_operation
_free_regions_at_end_of_collection = _g1h->num_free_regions();
_old_gen_alloc_tracker.reset_after_gc(_g1h->humongous_regions_count() * G1HeapRegion::GrainBytes);
Pause this_pause = collector_state()->gc_pause_type(concurrent_operation_is_full_mark);
size_t humongous_allocation_bytes = G1CollectedHeap::is_humongous(allocation_word_size) ?
G1CollectedHeap::allocation_used_bytes(allocation_word_size) : 0;
record_pause(this_pause, start_time_sec, end_time_sec, humongous_allocation_bytes);
// Do not update dynamic IHOP due to G1 periodic collection as it is highly likely
// that in this case we are not running in a "normal" operating mode.
if (_g1h->gc_cause() != GCCause::_g1_periodic_collection) {
update_young_length_bounds();
// Take snapshots of these values here as update_ihop_prediction
// may complete the concurrent cycle and reset the values.
size_t non_humongous_allocation = _concurrent_cycle_tracker.non_humongous_allocated_bytes();
size_t peak_extra_humongous_occupancy = _concurrent_cycle_tracker.peak_extra_humongous_occupancy_bytes();
if (update_ihop_prediction(app_time_ms / 1000.0, is_young_only_pause)) {
_ihop_control->report_statistics(_g1h->gc_tracer_stw(), _g1h->non_young_occupancy_after_allocation(allocation_word_size));
_ihop_control->report_statistics(_g1h->gc_tracer_stw(),
_g1h->non_young_occupancy_after_allocation(allocation_word_size),
non_humongous_allocation,
peak_extra_humongous_occupancy);
}
} else {
// Any garbage collection triggered as periodic collection resets the time-to-mixed
// measurement. Periodic collection typically means that the application is "inactive", i.e.
// the marking threads may have received an uncharacteristic amount of cpu time
// for completing the marking, i.e. are faster than expected.
// This skews the predicted marking length towards smaller values which might cause
// the mark start being too late.
abort_time_to_mixed_tracking();
}
// Note that _mmu_tracker->max_gc_time() returns the time in seconds.
@ -1012,10 +1032,8 @@ G1CollectorState G1Policy::record_young_collection_end(bool concurrent_operation
return next_state;
}
G1IHOPControl* G1Policy::create_ihop_control(const G1OldGenAllocationTracker* old_gen_alloc_tracker,
const G1Predictions* predictor) {
G1IHOPControl* G1Policy::create_ihop_control(const G1Predictions* predictor) {
return new G1IHOPControl(G1IHOP,
old_gen_alloc_tracker,
G1UseAdaptiveIHOP,
predictor,
G1ReservePercent,
@ -1032,29 +1050,31 @@ bool G1Policy::update_ihop_prediction(double mutator_time_s,
double const min_valid_time = 1e-6;
bool report = false;
if (!this_gc_was_young_only && _concurrent_cycle_tracker.has_completed_cycle()) {
G1ConcurrentCycleStats cycle_stats = _concurrent_cycle_tracker.get_and_reset_cycle_stats();
if (!this_gc_was_young_only && _concurrent_start_to_mixed.has_result()) {
double marking_to_mixed_time = _concurrent_start_to_mixed.get_and_reset_last_marking_time();
assert(marking_to_mixed_time > 0.0,
"Concurrent start to mixed time must be larger than zero but is %.3f",
marking_to_mixed_time);
if (marking_to_mixed_time > min_valid_time) {
_ihop_control->add_marking_start_to_mixed_length(marking_to_mixed_time);
double concurrent_cycle_duration_s = cycle_stats._cycle_duration_s;
assert(concurrent_cycle_duration_s > 0.0,
"Time for Concurrent Start GC to the first Mixed GC must be larger than zero but is %.3f",
concurrent_cycle_duration_s);
if (concurrent_cycle_duration_s > min_valid_time) {
_ihop_control->record_concurrent_cycle(concurrent_cycle_duration_s,
cycle_stats._non_humongous_allocated_bytes,
cycle_stats._peak_extra_humongous_occupancy_bytes);
report = true;
}
}
// As an approximation for the young gc promotion rates during marking we use
// all of them. In many applications there are only a few if any young gcs during
// marking, which makes any prediction useless. This increases the accuracy of the
// prediction.
// The second clause prevents skewing the IHOP prediction with (typically) degenerate
// back-to-back young-gen-size samples.
if (this_gc_was_young_only && mutator_time_s > min_valid_time) {
// IHOP control wants to know the expected young gen length if it were not
// restrained by the heap reserve. Using the actual length would make the
// prediction too small and the limit the young gen every time we get to the
// predicted target occupancy.
size_t young_gen_size = young_list_desired_length() * G1HeapRegion::GrainBytes;
_ihop_control->update_allocation_info(mutator_time_s, young_gen_size);
_ihop_control->record_expected_young_gen_size(young_gen_size);
report = true;
}
@ -1291,11 +1311,11 @@ void G1Policy::decide_on_concurrent_start_pause() {
// Force concurrent start.
collector_state()->set_in_concurrent_start_gc();
// We might have ended up coming here about to start a mixed phase with a collection set
// active. The following remark might change the change the "evacuation efficiency" of
// active. The following remark might change the "evacuation efficiency" of
// the regions in this set, leading to failing asserts later.
// Since the concurrent cycle will recreate the collection set anyway, simply drop it here.
abandon_collection_set_candidates();
abort_time_to_mixed_tracking();
abort_concurrent_cycle_tracking();
log_debug(gc, ergo)("Initiate concurrent cycle (%s requested concurrent cycle)",
requester_for_mixed_abort(cause));
} else {
@ -1335,7 +1355,7 @@ void G1Policy::record_concurrent_mark_cleanup_end(bool has_rebuilt_remembered_se
}
if (!mixed_gc_pending) {
abort_time_to_mixed_tracking();
abort_concurrent_cycle_tracking();
log_debug(gc, ergo)("request young-only gcs (candidate old regions not available)");
}
if (mixed_gc_pending) {
@ -1370,7 +1390,8 @@ void G1Policy::update_gc_pause_time_ratios(Pause gc_type, double start_time_sec,
void G1Policy::record_pause(Pause gc_type,
double start,
double end) {
double end,
size_t humongous_allocation_bytes) {
// Manage the MMU tracker. For some reason it ignores Full GCs.
if (gc_type != Pause::Full) {
_mmu_tracker->add_pause(start, end);
@ -1378,49 +1399,26 @@ void G1Policy::record_pause(Pause gc_type,
update_gc_pause_time_ratios(gc_type, start, end);
update_time_to_mixed_tracking(gc_type, start, end);
size_t humongous_bytes = _g1h->humongous_regions_count() * G1HeapRegion::GrainBytes;
G1AllocationIntervalStats alloc_interval_stats = _old_gen_alloc_tracker.end_allocation_interval(humongous_bytes);
bool is_periodic_gc = _g1h->gc_cause() == GCCause::_g1_periodic_collection;
if (humongous_allocation_bytes > 0) {
// Record the humongous allocation that triggered the GC and attribute it to
// the ending allocation interval. We do this eagerly, before we know whether
// the post GC allocation succeeds, to keep the common case simple. In the
// rare case where this allocation fails, we over-account; this only
// affects the stored IHOP sample if the current GC is the first Mixed GC.
alloc_interval_stats.record_humongous_allocation(humongous_allocation_bytes);
}
_concurrent_cycle_tracker.record_allocation_interval(gc_type, is_periodic_gc, start, end, alloc_interval_stats);
double elapsed_gc_cpu_time = _analytics->gc_cpu_time_ms();
_analytics->set_gc_cpu_time_at_pause_end_ms(elapsed_gc_cpu_time);
}
void G1Policy::update_time_to_mixed_tracking(Pause gc_type,
double start,
double end) {
// Manage the mutator time tracking from concurrent start to first mixed gc.
switch (gc_type) {
case Pause::Full:
abort_time_to_mixed_tracking();
break;
case Pause::Cleanup:
case Pause::Remark:
case Pause::Normal:
case Pause::PrepareMixed:
_concurrent_start_to_mixed.add_pause(end - start);
break;
case Pause::ConcurrentStartFull:
// Do not track time-to-mixed time for periodic collections as they are likely
// to be not representative to regular operation as the mutators are idle at
// that time. Also only track full concurrent mark cycles.
if (_g1h->gc_cause() != GCCause::_g1_periodic_collection) {
_concurrent_start_to_mixed.record_concurrent_start_end(end);
}
break;
case Pause::ConcurrentStartUndo:
assert(_g1h->gc_cause() == GCCause::_g1_humongous_allocation,
"GC cause must be humongous allocation but is %d",
_g1h->gc_cause());
break;
case Pause::Mixed:
_concurrent_start_to_mixed.record_mixed_gc_start(start);
break;
default:
ShouldNotReachHere();
}
}
void G1Policy::abort_time_to_mixed_tracking() {
_concurrent_start_to_mixed.reset();
void G1Policy::abort_concurrent_cycle_tracking() {
_concurrent_cycle_tracker.abort_cycle();
}
bool G1Policy::next_gc_should_be_mixed() const {

33
src/hotspot/share/gc/g1/g1Policy.hpp
View File

@ -26,7 +26,7 @@
#define SHARE_GC_G1_G1POLICY_HPP
#include "gc/g1/g1CollectorState.hpp"
#include "gc/g1/g1ConcurrentStartToMixedTimeTracker.hpp"
#include "gc/g1/g1ConcurrentCycleTracker.hpp"
#include "gc/g1/g1GCPhaseTimes.hpp"
#include "gc/g1/g1HeapRegionAttr.hpp"
#include "gc/g1/g1MMUTracker.hpp"
@ -58,8 +58,7 @@ class STWGCTimer;
class G1Policy: public CHeapObj<mtGC> {
using Pause = G1CollectorState::Pause;
static G1IHOPControl* create_ihop_control(const G1OldGenAllocationTracker* old_gen_alloc_tracker,
const G1Predictions* predictor);
static G1IHOPControl* create_ihop_control(const G1Predictions* predictor);
// Update the IHOP control with the necessary statistics. Returns true if there
// has been a significant update to the prediction.
bool update_ihop_prediction(double mutator_time_s,
@ -70,8 +69,9 @@ class G1Policy: public CHeapObj<mtGC> {
G1RemSetTrackingPolicy _remset_tracker;
G1MMUTracker* _mmu_tracker;
G1ConcurrentCycleTracker _concurrent_cycle_tracker;
// Tracking the allocation in the old generation between
// two GCs.
// two pauses.
G1OldGenAllocationTracker _old_gen_alloc_tracker;
G1IHOPControl* _ihop_control;
@ -91,9 +91,11 @@ class G1Policy: public CHeapObj<mtGC> {
G1SurvRateGroup* _survivor_surv_rate_group;
double _reserve_factor;
// This will be set when the heap is expanded
// for the first time during initialization.
uint _reserve_regions;
// The allocation reserve in number of regions that we try to keep free.
// G1 allocation of new regions for eden is restrained when allocating into that reserve.
// This intentionally slows down the allocation when the heap is close to full to allow
// concurrent marking to finish and hopefully avoid a Full GC.
Atomic<uint> _reserve_regions;
G1YoungGenSizer _young_gen_sizer;
@ -112,8 +114,6 @@ class G1Policy: public CHeapObj<mtGC> {
// garbage collection or the most recent refinement sweep.
size_t _to_collection_set_cards;
G1ConcurrentStartToMixedTimeTracker _concurrent_start_to_mixed;
bool should_update_surv_rate_group_predictors();
double pending_cards_processing_time() const;
@ -224,9 +224,13 @@ private:
// Calculate desired young length based on current situation without taking actually
// available free regions into account.
uint calculate_young_desired_length(size_t pending_cards, size_t card_rs_length, size_t code_root_rs_length) const;
uint calculate_young_desired_length(size_t pending_cards,
size_t card_rs_length,
size_t code_root_rs_length,
uint min_young_length_by_sizer,
uint max_young_length_by_sizer) const;
// Limit the given desired young length to available free regions.
uint calculate_young_target_length(uint desired_young_length) const;
uint calculate_young_target_length(uint desired_young_length, uint min_young_length_by_sizer) const;
double predict_survivor_regions_evac_time() const;
double predict_retained_regions_evac_time() const;
@ -258,17 +262,16 @@ public:
private:
void abandon_collection_set_candidates();
// Manage time-to-mixed tracking.
void update_time_to_mixed_tracking(Pause gc_type, double start, double end);
// Record the given STW pause with the given start and end times (in s).
void record_pause(Pause gc_type,
double start,
double end);
double end,
size_t humongous_allocation_bytes = 0);
void update_gc_pause_time_ratios(Pause gc_type, double start_sec, double end_sec);
// Indicate that we aborted marking before doing any mixed GCs.
void abort_time_to_mixed_tracking();
void abort_concurrent_cycle_tracking();
public:

43
src/hotspot/share/gc/g1/g1Trace.cpp
View File

@ -97,31 +97,31 @@ void G1NewTracer::report_evacuation_statistics(const G1EvacSummary& young_summar
}
void G1NewTracer::report_basic_ihop_statistics(size_t threshold,
size_t target_ccupancy,
size_t non_young_occupancy,
size_t last_allocation_size,
double last_allocation_duration,
double last_marking_length) {
size_t target_occupancy,
size_t non_young_occupancy) {
send_basic_ihop_statistics(threshold,
target_ccupancy,
non_young_occupancy,
last_allocation_size,
last_allocation_duration,
last_marking_length);
target_occupancy,
non_young_occupancy);
}
void G1NewTracer::report_adaptive_ihop_statistics(size_t threshold,
size_t internal_target_occupancy,
size_t current_occupancy,
size_t additional_buffer_size,
double predicted_allocation_rate,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy,
double predicted_old_non_hum_alloc_rate,
size_t predicted_peak_extra_humongous_occupancy,
double predicted_marking_length,
bool prediction_active) {
send_adaptive_ihop_statistics(threshold,
internal_target_occupancy,
current_occupancy,
additional_buffer_size,
predicted_allocation_rate,
non_humongous_allocation,
peak_extra_humongous_occupancy,
predicted_old_non_hum_alloc_rate,
predicted_peak_extra_humongous_occupancy,
predicted_marking_length,
prediction_active);
}
@ -206,10 +206,7 @@ void G1NewTracer::send_old_evacuation_statistics(const G1EvacSummary& summary) c
void G1NewTracer::send_basic_ihop_statistics(size_t threshold,
size_t target_occupancy,
size_t non_young_occupancy,
size_t last_allocation_size,
double last_allocation_duration,
double last_marking_length) {
size_t non_young_occupancy) {
EventG1BasicIHOP evt;
if (evt.should_commit()) {
evt.set_gcId(GCId::current());
@ -217,10 +214,6 @@ void G1NewTracer::send_basic_ihop_statistics(size_t threshold,
evt.set_targetOccupancy(target_occupancy);
evt.set_thresholdPercentage(target_occupancy > 0 ? ((double)threshold / target_occupancy) : 0.0);
evt.set_currentOccupancy(non_young_occupancy);
evt.set_recentMutatorAllocationSize(last_allocation_size);
evt.set_recentMutatorDuration(last_allocation_duration * MILLIUNITS);
evt.set_recentAllocationRate(last_allocation_duration != 0.0 ? last_allocation_size / last_allocation_duration : 0.0);
evt.set_lastMarkingDuration(last_marking_length * MILLIUNITS);
evt.commit();
}
}
@ -229,7 +222,10 @@ void G1NewTracer::send_adaptive_ihop_statistics(size_t threshold,
size_t internal_target_occupancy,
size_t current_occupancy,
size_t additional_buffer_size,
double predicted_allocation_rate,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy,
double predicted_old_non_hum_alloc_rate,
size_t predicted_peak_extra_humongous_occupancy,
double predicted_marking_length,
bool prediction_active) {
EventG1AdaptiveIHOP evt;
@ -240,7 +236,10 @@ void G1NewTracer::send_adaptive_ihop_statistics(size_t threshold,
evt.set_ihopTargetOccupancy(internal_target_occupancy);
evt.set_currentOccupancy(current_occupancy);
evt.set_additionalBufferSize(additional_buffer_size);
evt.set_predictedAllocationRate(predicted_allocation_rate);
evt.set_nonHumongousAllocation(non_humongous_allocation);
evt.set_peakExtraHumongousOccupancy(peak_extra_humongous_occupancy);
evt.set_predictedNonHumongousAllocation(predicted_old_non_hum_alloc_rate);
evt.set_predictedPeakExtraHumongousOccupancy(predicted_peak_extra_humongous_occupancy);
evt.set_predictedMarkingDuration(predicted_marking_length * MILLIUNITS);
evt.set_predictionActive(prediction_active);
evt.commit();

21
src/hotspot/share/gc/g1/g1Trace.hpp
View File

@ -52,15 +52,15 @@ public:
void report_basic_ihop_statistics(size_t threshold,
size_t target_occupancy,
size_t current_occupancy,
size_t last_allocation_size,
double last_allocation_duration,
double last_marking_length);
size_t current_occupancy);
void report_adaptive_ihop_statistics(size_t threshold,
size_t internal_target_occupancy,
size_t current_occupancy,
size_t additional_buffer_size,
double predicted_allocation_rate,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy,
double predicted_old_gen_non_humongous_allocation_rate,
size_t predicted_peak_extra_humongous_occupancy,
double predicted_marking_length,
bool prediction_active);
private:
@ -73,15 +73,16 @@ private:
void send_basic_ihop_statistics(size_t threshold,
size_t target_occupancy,
size_t non_young_occupancy,
size_t last_allocation_size,
double last_allocation_duration,
double last_marking_length);
size_t non_young_occupancy);
void send_adaptive_ihop_statistics(size_t threshold,
size_t internal_target_occupancy,
size_t non_young_occupancy,
size_t additional_buffer_size,
double predicted_allocation_rate,
size_t non_humongous_allocation,
size_t peak_extra_humongous_occupancy,
double predicted_old_gen_non_humongous_allocation_rate,
size_t predicted_peak_extra_humongous_occupancy,
double predicted_marking_length,
bool prediction_active);
};

6
src/hotspot/share/gc/g1/g1UncommitRegionTask.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2020, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -58,9 +58,9 @@ void G1UncommitRegionTask::enqueue() {
G1UncommitRegionTask* uncommit_task = instance();
if (!uncommit_task->is_active()) {
// Change state to active and schedule using UncommitInitialDelayMs.
// Change state to active and schedule.
uncommit_task->set_active(true);
G1CollectedHeap::heap()->service_thread()->schedule_task(uncommit_task, UncommitInitialDelayMs);
G1CollectedHeap::heap()->service_thread()->schedule_task(uncommit_task, G1UncommitInitialDelay);
}
}

4
src/hotspot/share/gc/g1/g1UncommitRegionTask.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2020, 2022, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2020, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -34,8 +34,6 @@ class G1UncommitRegionTask : public G1ServiceTask {
// This limit is small enough to ensure that the duration of each invocation
// is short, while still making reasonable progress.
static const uint UncommitSizeLimit = 128 * M;
// Initial delay in milliseconds after GC before the regions are uncommitted.
static const uint UncommitInitialDelayMs = 100;
// The delay between two uncommit task executions.
static const uint UncommitTaskDelayMs = 10;

10
src/hotspot/share/gc/g1/g1YoungGCPostEvacuateTasks.cpp
View File

@ -542,7 +542,7 @@ public:
class FreeCSetStats {
size_t _before_used_bytes; // Usage in regions successfully evacuate
size_t _after_used_bytes; // Usage in regions failing evacuation
size_t _bytes_allocated_in_old_since_last_gc; // Size of young regions turned into old
size_t _bytes_allocated_in_old_since_last_pause; // Size of young regions turned into old
size_t _failure_used_words; // Live size in failed regions
size_t _failure_waste_words; // Wasted size in failed regions
uint _regions_freed; // Number of regions freed
@ -551,7 +551,7 @@ public:
FreeCSetStats() :
_before_used_bytes(0),
_after_used_bytes(0),
_bytes_allocated_in_old_since_last_gc(0),
_bytes_allocated_in_old_since_last_pause(0),
_failure_used_words(0),
_failure_waste_words(0),
_regions_freed(0) { }
@ -560,7 +560,7 @@ public:
assert(other != nullptr, "invariant");
_before_used_bytes += other->_before_used_bytes;
_after_used_bytes += other->_after_used_bytes;
_bytes_allocated_in_old_since_last_gc += other->_bytes_allocated_in_old_since_last_gc;
_bytes_allocated_in_old_since_last_pause += other->_bytes_allocated_in_old_since_last_pause;
_failure_used_words += other->_failure_used_words;
_failure_waste_words += other->_failure_waste_words;
_regions_freed += other->_regions_freed;
@ -575,7 +575,7 @@ public:
g1h->alloc_buffer_stats(G1HeapRegionAttr::Old)->add_failure_used_and_waste(_failure_used_words, _failure_waste_words);
G1Policy *policy = g1h->policy();
policy->old_gen_alloc_tracker()->add_allocated_bytes_since_last_gc(_bytes_allocated_in_old_since_last_gc);
policy->old_gen_alloc_tracker()->add_allocated_non_humongous_bytes(_bytes_allocated_in_old_since_last_pause);
policy->cset_regions_freed();
}
@ -592,7 +592,7 @@ public:
// additional allocation: both the objects still in the region and the
// ones already moved are accounted for elsewhere.
if (r->is_young()) {
_bytes_allocated_in_old_since_last_gc += G1HeapRegion::GrainBytes;
_bytes_allocated_in_old_since_last_pause += G1HeapRegion::GrainBytes;
}
}

26
src/hotspot/share/gc/g1/g1YoungGenSizer.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2016, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2016, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -30,7 +30,7 @@
#include "runtime/globals_extension.hpp"
G1YoungGenSizer::G1YoungGenSizer() : _sizer_kind(SizerDefaults),
_use_adaptive_sizing(true), _min_desired_young_length(0), _max_desired_young_length(0) {
_use_adaptive_sizing(true), _min_desired_young_length(), _max_desired_young_length(0) {
precond(!FLAG_IS_ERGO(NewRatio));
precond(!FLAG_IS_ERGO(NewSize));
@ -100,16 +100,16 @@ G1YoungGenSizer::G1YoungGenSizer() : _sizer_kind(SizerDefaults),
}
if (user_specified_NewSize) {
_min_desired_young_length = MAX2((uint)(NewSize / G1HeapRegion::GrainBytes), 1U);
_min_desired_young_length.store_relaxed(MAX2((uint)(NewSize / G1HeapRegion::GrainBytes), 1U));
}
if (user_specified_MaxNewSize) {
_max_desired_young_length = MAX2((uint)(MaxNewSize / G1HeapRegion::GrainBytes), 1U);
_max_desired_young_length.store_relaxed(MAX2((uint)(MaxNewSize / G1HeapRegion::GrainBytes), 1U));
}
if (user_specified_NewSize && user_specified_MaxNewSize) {
_sizer_kind = SizerMaxAndNewSize;
_use_adaptive_sizing = _min_desired_young_length != _max_desired_young_length;
_use_adaptive_sizing = min_desired_young_length() != max_desired_young_length();
} else if (user_specified_NewSize) {
_sizer_kind = SizerNewSizeOnly;
} else {
@ -159,20 +159,22 @@ void G1YoungGenSizer::recalculate_min_max_young_length(uint number_of_heap_regio
}
void G1YoungGenSizer::adjust_max_new_size(uint number_of_heap_regions) {
// We need to pass the desired values because recalculation may not update these
// values in some cases.
uint temp = _min_desired_young_length;
uint result = _max_desired_young_length;
recalculate_min_max_young_length(number_of_heap_regions, &temp, &result);
uint unused_new_min = min_desired_young_length();
uint new_max = max_desired_young_length();
recalculate_min_max_young_length(number_of_heap_regions, &unused_new_min, &new_max);
size_t max_young_size = result * G1HeapRegion::GrainBytes;
size_t max_young_size = new_max * G1HeapRegion::GrainBytes;
if (max_young_size != MaxNewSize) {
FLAG_SET_ERGO(MaxNewSize, max_young_size);
}
}
void G1YoungGenSizer::heap_size_changed(uint new_number_of_heap_regions) {
recalculate_min_max_young_length(new_number_of_heap_regions, &_min_desired_young_length,
&_max_desired_young_length);
uint min = min_desired_young_length();
uint max = max_desired_young_length();
recalculate_min_max_young_length(new_number_of_heap_regions, &min, &max);
_min_desired_young_length.store_relaxed(min);
_max_desired_young_length.store_relaxed(max);
}

11
src/hotspot/share/gc/g1/g1YoungGenSizer.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2016, 2022, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2016, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -25,6 +25,7 @@
#ifndef SHARE_GC_G1_G1YOUNGGENSIZER_HPP
#define SHARE_GC_G1_G1YOUNGGENSIZER_HPP
#include "runtime/atomic.hpp"
#include "utilities/globalDefinitions.hpp"
// There are three command line options related to the young gen size:
@ -78,8 +79,8 @@ private:
// true otherwise.
bool _use_adaptive_sizing;
uint _min_desired_young_length;
uint _max_desired_young_length;
Atomic<uint> _min_desired_young_length;
Atomic<uint> _max_desired_young_length;
uint calculate_default_min_length(uint new_number_of_heap_regions);
uint calculate_default_max_length(uint new_number_of_heap_regions);
@ -96,10 +97,10 @@ public:
virtual void heap_size_changed(uint new_number_of_heap_regions);
uint min_desired_young_length() const {
return _min_desired_young_length;
return _min_desired_young_length.load_relaxed();
}
uint max_desired_young_length() const {
return _max_desired_young_length;
return _max_desired_young_length.load_relaxed();
}
bool use_adaptive_young_list_length() const {

6
src/hotspot/share/gc/g1/g1_globals.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2001, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2001, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -184,6 +184,10 @@
"shrink attempt.") \
range(0, 100) \
\
develop(uint, G1UncommitInitialDelay, 100, \
"Delay in milliseconds until regions just made eligible for " \
"uncommit are actually uncommitted.") \
\
product(uint, G1CPUUsageDeviationPercent, 25, DIAGNOSTIC, \
"The acceptable deviation (in percent) from the target GC CPU " \
"usage (based on GCTimeRatio). Creates a tolerance range " \

13
src/hotspot/share/gc/serial/cardTableRS.cpp
View File

@ -49,7 +49,14 @@ void CardTableRS::scan_old_to_young_refs(TenuredGeneration* tg, HeapWord* saved_
void CardTableRS::maintain_old_to_young_invariant(TenuredGeneration* old_gen,
bool is_young_gen_empty) {
if (is_young_gen_empty) {
clear_MemRegion(old_gen->prev_used_region());
MemRegion prev_used_mr = old_gen->prev_used_region();
if (!prev_used_mr.is_empty()) {
clear_MemRegion(prev_used_mr);
}
{
MemRegion old_gen_committed{old_gen->space()->bottom(), old_gen->space()->end()};
verify_region(old_gen_committed, clean_card_val(), true);
}
} else {
MemRegion used_mr = old_gen->used_region();
MemRegion prev_used_mr = old_gen->prev_used_region();
@ -59,7 +66,9 @@ void CardTableRS::maintain_old_to_young_invariant(TenuredGeneration* old_gen,
}
// No idea which card contains old-to-young pointer, so dirtying cards for
// the entire used part of old-gen conservatively.
dirty_MemRegion(used_mr);
if (!used_mr.is_empty()) {
dirty_MemRegion(used_mr);
}
}
}

12
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp
View File

@ -27,7 +27,6 @@
#include "gc/shared/barrierSet.hpp"
#include "gc/shenandoah/c2/shenandoahBarrierSetC2.hpp"
#include "gc/shenandoah/heuristics/shenandoahHeuristics.hpp"
#include "gc/shenandoah/shenandoahForwarding.hpp"
#include "gc/shenandoah/shenandoahHeap.hpp"
#include "gc/shenandoah/shenandoahRuntime.hpp"
#include "gc/shenandoah/shenandoahThreadLocalData.hpp"
@ -712,10 +711,9 @@ void ShenandoahBarrierSetC2::print_barrier_data(outputStream* os, uint8_t data)
fatal("Unknown bit!");
}
os->print_cr(" GC configuration: %sLRB %sSATB %sCAS %sClone %sCard",
os->print_cr(" GC configuration: %sLRB %sSATB %sClone %sCard",
(ShenandoahLoadRefBarrier ? "+" : "-"),
(ShenandoahSATBBarrier ? "+" : "-"),
(ShenandoahCASBarrier ? "+" : "-"),
(ShenandoahCloneBarrier ? "+" : "-"),
(ShenandoahCardBarrier ? "+" : "-")
);
@ -912,16 +910,16 @@ void ShenandoahBarrierStubC2::load_post(MacroAssembler* masm, const MachNode* no
}
}
void ShenandoahBarrierStubC2::store_pre(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow) {
void ShenandoahBarrierStubC2::store_pre(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2, Register tmp3, bool narrow) {
// Store pre-barrier: SATB, keep-alive the current memory value.
if (needs_slow_barrier(node)) {
assert(!needs_load_ref_barrier(node), "Should not be required for stores");
ShenandoahBarrierStubC2* const stub = create(node, obj, addr, tmp1, tmp2, narrow, /* do_load = */ true);
ShenandoahBarrierStubC2* const stub = create(node, tmp1, addr, tmp2, tmp3, narrow, /* do_load = */ true);
stub->enter_if_gc_state(*masm, ShenandoahHeap::MARKING, tmp1);
}
}
void ShenandoahBarrierStubC2::load_store_pre(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow) {
void ShenandoahBarrierStubC2::load_store_pre(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2, Register tmp3, bool narrow) {
// Load/Store pre-barrier:
// a. Avoids false positives from CAS encountering to-space memory values.
// b. Satisfies the need for LRB for the CAE result.
@ -930,7 +928,7 @@ void ShenandoahBarrierStubC2::load_store_pre(MacroAssembler* masm, const MachNod
// (a) and (b) are covered because load barrier does memory location fixup.
// (c) is covered by KA on the current memory value.
if (needs_slow_barrier(node)) {
ShenandoahBarrierStubC2* const stub = create(node, obj, addr, tmp1, tmp2, narrow, /* do_load = */ true);
ShenandoahBarrierStubC2* const stub = create(node, tmp1, addr, tmp2, tmp3, narrow, /* do_load = */ true);
char check = 0;
check |= needs_keep_alive_barrier(node) ? ShenandoahHeap::MARKING : 0;
check |= needs_load_ref_barrier(node) ? ShenandoahHeap::HAS_FORWARDED : 0;

8
src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.hpp
View File

@ -228,10 +228,10 @@ public:
return needs_load_ref_barrier(node) || needs_keep_alive_barrier(node);
}
static void load_post(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow);
static void store_pre(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow);
static void store_post(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2);
static void load_store_pre(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow);
static void load_post(MacroAssembler* masm, const MachNode* node, Register obj, Address addr, Register tmp1, Register tmp2, bool narrow);
static void store_pre(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2, Register tmp3, bool narrow);
static void store_post(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2);
static void load_store_pre(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2, Register tmp3, bool narrow);
static void load_store_post(MacroAssembler* masm, const MachNode* node, Address addr, Register tmp1, Register tmp2);
void emit_code(MacroAssembler& masm);

1
src/hotspot/share/gc/shenandoah/mode/shenandoahGenerationalMode.cpp
View File

@ -57,7 +57,6 @@ void ShenandoahGenerationalMode::initialize_flags() const {
// Final configuration checks
SHENANDOAH_CHECK_FLAG_SET(ShenandoahLoadRefBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahSATBBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahCASBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahCloneBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahCardBarrier);
}

1
src/hotspot/share/gc/shenandoah/mode/shenandoahPassiveMode.cpp
View File

@ -48,7 +48,6 @@ void ShenandoahPassiveMode::initialize_flags() const {
// Disable known barriers by default.
SHENANDOAH_ERGO_DISABLE_FLAG(ShenandoahLoadRefBarrier);
SHENANDOAH_ERGO_DISABLE_FLAG(ShenandoahSATBBarrier);
SHENANDOAH_ERGO_DISABLE_FLAG(ShenandoahCASBarrier);
SHENANDOAH_ERGO_DISABLE_FLAG(ShenandoahCloneBarrier);
SHENANDOAH_ERGO_DISABLE_FLAG(ShenandoahCardBarrier);
}

1
src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp
View File

@ -40,7 +40,6 @@ void ShenandoahSATBMode::initialize_flags() const {
// Final configuration checks
SHENANDOAH_CHECK_FLAG_SET(ShenandoahLoadRefBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahSATBBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahCASBarrier);
SHENANDOAH_CHECK_FLAG_SET(ShenandoahCloneBarrier);
SHENANDOAH_CHECK_FLAG_UNSET(ShenandoahCardBarrier);
}

1
src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp
View File

@ -63,7 +63,6 @@ void ShenandoahArguments::initialize() {
FLAG_SET_DEFAULT(ShenandoahSATBBarrier, false);
FLAG_SET_DEFAULT(ShenandoahLoadRefBarrier, false);
FLAG_SET_DEFAULT(ShenandoahCASBarrier, false);
FLAG_SET_DEFAULT(ShenandoahCardBarrier, false);
FLAG_SET_DEFAULT(ShenandoahCloneBarrier, false);

12
src/hotspot/share/gc/shenandoah/shenandoahClosures.hpp
View File

@ -253,4 +253,16 @@ public:
};
#endif // ASSERT
class ShenandoahMultiThreadClosure : public ThreadClosure {
ThreadClosure& _cl1;
ThreadClosure& _cl2;
public:
ShenandoahMultiThreadClosure(ThreadClosure& cl1, ThreadClosure& cl2) :
_cl1(cl1), _cl2(cl2) {}
inline void do_thread(Thread* thread) override {
_cl1.do_thread(thread);
_cl2.do_thread(thread);
}
};
#endif // SHARE_GC_SHENANDOAH_SHENANDOAHCLOSURES_HPP

107
src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.cpp
View File

@ -59,6 +59,10 @@ public:
ShenandoahWorkerTimingsTracker timer(ShenandoahPhaseTimings::conc_mark, ShenandoahPhaseTimings::Work, worker_id, true);
SuspendibleThreadSetJoiner stsj;
_cm->mark_loop(worker_id, _terminator, GENERATION, true /*cancellable*/);
// Concurrent marking loop flushes Java thread buffers, coordinating with a handshake.
// Here, a GC worker has completed marking work, so it is a good time to flush its SATB buffers too.
SATBMarkQueueSet& satb_mq_set = ShenandoahBarrierSet::satb_mark_queue_set();
satb_mq_set.flush_queue(ShenandoahThreadLocalData::satb_mark_queue(Thread::current()));
}
};
@ -67,32 +71,14 @@ class ShenandoahFinalMarkingTask : public WorkerTask {
private:
ShenandoahConcurrentMark* _cm;
TaskTerminator* _terminator;
ThreadsClaimTokenScope _threads_claim_token_scope; // needed for Threads::possibly_parallel_threads_do
public:
ShenandoahFinalMarkingTask(ShenandoahConcurrentMark* cm, TaskTerminator* terminator) :
WorkerTask("Shenandoah Final Mark"), _cm(cm), _terminator(terminator),
_threads_claim_token_scope() {
}
WorkerTask("Shenandoah Final Mark"), _cm(cm), _terminator(terminator) {}
void work(uint worker_id) {
ShenandoahHeap* heap = ShenandoahHeap::heap();
ShenandoahWorkerTimingsTracker timer(ShenandoahPhaseTimings::finish_mark, ShenandoahPhaseTimings::Work, worker_id, true);
ShenandoahParallelWorkerSession worker_session(worker_id);
// First drain remaining SATB buffers.
{
ShenandoahObjToScanQueue* q = _cm->get_queue(worker_id);
ShenandoahObjToScanQueue* old_q = _cm->get_old_queue(worker_id);
ShenandoahSATBBufferClosure<GENERATION> cl(q, old_q);
SATBMarkQueueSet& satb_mq_set = ShenandoahBarrierSet::satb_mark_queue_set();
while (satb_mq_set.apply_closure_to_completed_buffer(&cl)) {}
assert(!heap->has_forwarded_objects(), "Not expected");
ShenandoahFlushSATB tc(satb_mq_set);
Threads::possibly_parallel_threads_do(true /* is_par */, &tc);
}
_cm->mark_loop(worker_id, _terminator, GENERATION, false /*not cancellable*/);
assert(_cm->task_queues()->is_empty(), "Should be empty");
}
@ -255,49 +241,54 @@ void ShenandoahConcurrentMark::finish_mark() {
}
void ShenandoahConcurrentMark::finish_mark_work() {
// Finally mark everything else we've got in our queues during the previous steps.
// It does two different things for concurrent vs. mark-compact GC:
// - For concurrent GC, it starts with empty task queues, drains the remaining
// SATB buffers, and then completes the marking closure.
// - For mark-compact GC, it starts out with the task queues seeded by initial
// root scan, and completes the closure, thus marking through all live objects
// The implementation is the same, so it's shared here.
ShenandoahHeap* const heap = ShenandoahHeap::heap();
ShenandoahGCPhase phase(ShenandoahPhaseTimings::finish_mark);
uint nworkers = heap->workers()->active_workers();
task_queues()->reserve(nworkers);
SATBMarkQueueSet& satb_mq_set = ShenandoahBarrierSet::satb_mark_queue_set();
TaskTerminator terminator(nworkers, task_queues());
switch (_generation->type()) {
case YOUNG:{
ShenandoahFinalMarkingTask<YOUNG> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case OLD:{
ShenandoahFinalMarkingTask<OLD> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case GLOBAL:{
ShenandoahFinalMarkingTask<GLOBAL> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case NON_GEN:{
ShenandoahFinalMarkingTask<NON_GEN> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
default:
ShouldNotReachHere();
// First drain all remaining SATB buffers and put them to SATB MQ.
// Also, while we are iterating threads, mark the invisible roots.
{
ShenandoahTimingsTracker t(ShenandoahPhaseTimings::final_mark_flush_satb_roots);
ShenandoahInvisibleRootsMarkClosure invisible_cl;
ShenandoahFlushSATB flush_cl(satb_mq_set);
ShenandoahMultiThreadClosure mux(flush_cl, invisible_cl);
Threads::threads_do(&mux);
}
if (!generation()->is_old() && heap->is_concurrent_young_mark_in_progress()) {
// Lastly, ensure all the invisible roots are marked.
ShenandoahInvisibleRootsMarkClosure cl;
Threads::java_threads_do(&cl);
// There is a very high chance we have already completed the marking.
// But if there is outstanding work, finish it now.
if (!task_queues()->is_empty() || satb_mq_set.completed_buffers_num() > 0) {
ShenandoahGCPhase phase(ShenandoahPhaseTimings::finish_mark);
uint nworkers = heap->workers()->active_workers();
task_queues()->reserve(nworkers);
TaskTerminator terminator(nworkers, task_queues());
switch (_generation->type()) {
case YOUNG:{
ShenandoahFinalMarkingTask<YOUNG> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case OLD:{
ShenandoahFinalMarkingTask<OLD> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case GLOBAL:{
ShenandoahFinalMarkingTask<GLOBAL> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
case NON_GEN:{
ShenandoahFinalMarkingTask<NON_GEN> task(this, &terminator);
heap->workers()->run_task(&task);
break;
}
default:
ShouldNotReachHere();
}
}
assert(task_queues()->is_empty(), "Should be empty");
assert(satb_mq_set.completed_buffers_num() == 0, "Should be empty");
}

1
src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp
View File

@ -70,6 +70,7 @@ class outputStream;
SHENANDOAH_SIMPLE_PHASE_DEF(f, final_mark_gross, "Pause Final Mark (G)") \
SHENANDOAH_SIMPLE_PHASE_DEF(f, final_mark, "Pause Final Mark (N)") \
SHENANDOAH_SIMPLE_PHASE_DEF(f, final_mark_verify, " Verify") \
SHENANDOAH_SIMPLE_PHASE_DEF(f, final_mark_flush_satb_roots, " Flush SATB and Roots") \
SHENANDOAH_WORKER_PHASE_DEF(f, finish_mark, " Finish Mark", \
" FM: ") \
SHENANDOAH_SIMPLE_PHASE_DEF(f, final_mark_propagate_gc_state, " Propagate GC State") \

3
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp
View File

@ -500,9 +500,6 @@
"Turn on/off card-marking post-write barrier in Shenandoah: " \
" true when ShenandoahGCMode is generational, false otherwise") \
\
product(bool, ShenandoahCASBarrier, true, DIAGNOSTIC, \
"Turn on/off CAS barriers in Shenandoah") \
\
product(bool, ShenandoahCloneBarrier, true, DIAGNOSTIC, \
"Turn on/off clone barriers in Shenandoah") \
\

106
src/hotspot/share/jfr/dcmd/jfrDcmds.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2012, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -27,6 +27,7 @@
#include "jfr/dcmd/jfrDcmds.hpp"
#include "jfr/jfr.hpp"
#include "jfr/jni/jfrJavaSupport.hpp"
#include "jfr/periodic/jfrRedactedEvents.hpp"
#include "jfr/recorder/jfrRecorder.hpp"
#include "jfr/recorder/service/jfrOptionSet.hpp"
#include "jfr/support/jfrThreadLocal.hpp"
@ -401,11 +402,25 @@ JfrConfigureFlightRecorderDCmd::JfrConfigureFlightRecorderDCmd(outputStream* out
};
void JfrConfigureFlightRecorderDCmd::print_help(const char* name) const {
outputStream* out = output();
print_help(output(), false);
}
static void print_filters(outputStream* out, JfrRedactedEvents::StringArray* filters) {
for (int i = 0; i < filters->length(); i++) {
out->print_cr(" %s", filters->at(i)->text());
}
}
void JfrConfigureFlightRecorderDCmd::print_help(outputStream* out, bool startup) {
// 0123456789001234567890012345678900123456789001234567890012345678900123456789001234567890
if (startup) {
out->print_cr("Syntax : -XX:FlightRecorderOptions:[options]");
out->print_cr("");
}
out->print_cr("Options:");
out->print_cr("");
out->print_cr(" globalbuffercount (Optional) Number of global buffers. This option is a legacy");
out->print_cr( " %-19s (Optional) Number of global buffers. This option is a legacy",
startup ? "numglobalbuffers": "globalbuffercount");
out->print_cr(" option: change the memorysize parameter to alter the number of");
out->print_cr(" global buffers. This value cannot be changed once JFR has been");
out->print_cr(" initialized. (STRING, default determined by the value for");
@ -427,7 +442,8 @@ void JfrConfigureFlightRecorderDCmd::print_help(const char* name) const {
out->print_cr(" gigabytes. This value cannot be changed once JFR has been");
out->print_cr(" initialized. (STRING, 10M)");
out->print_cr("");
out->print_cr(" repositorypath (Optional) Path to the location where recordings are stored until");
out->print_cr( " %-19s (Optional) Path to the location where recordings are stored until",
startup ? "repository" : "repositorypath");
out->print_cr(" they are written to a permanent file. (STRING, The default");
out->print_cr(" location is the temporary directory for the operating system. On");
out->print_cr(" Linux operating systems, the temporary directory is /tmp. On");
@ -443,7 +459,8 @@ void JfrConfigureFlightRecorderDCmd::print_help(const char* name) const {
out->print_cr(" degradation. This value cannot be changed once JFR has been");
out->print_cr(" initialized. (LONG, 64)");
out->print_cr("");
out->print_cr(" thread_buffer_size (Optional) Local buffer size for each thread in bytes if one of");
out->print_cr( " %-19s (Optional) Local buffer size for each thread in bytes if one of",
startup ? "threadbuffersize" : "thread_buffer_size");
out->print_cr(" the following suffixes is not used: 'k' or 'K' for kilobytes or");
out->print_cr(" 'm' or 'M' for megabytes. Overriding this parameter could reduce");
out->print_cr(" performance and is not recommended. This value cannot be changed");
@ -452,14 +469,77 @@ void JfrConfigureFlightRecorderDCmd::print_help(const char* name) const {
out->print_cr(" preserve-repository (Optional) Preserve files stored in the disk repository after the");
out->print_cr(" Java Virtual Machine has exited. (BOOLEAN, false)");
out->print_cr("");
out->print_cr("Options must be specified using the <key> or <key>=<value> syntax.");
out->print_cr("");
out->print_cr("Example usage:");
out->print_cr("");
out->print_cr(" $ jcmd <pid> JFR.configure");
out->print_cr(" $ jcmd <pid> JFR.configure repositorypath=/temporary");
out->print_cr(" $ jcmd <pid> JFR.configure stackdepth=256");
out->print_cr(" $ jcmd <pid> JFR.configure memorysize=100M");
if (startup) {
out->print_cr(" old-object-queue-size (Optional) Maximum number of old objects to track. By default,");
out->print_cr(" the number of objects is set to 256. (LONG, 256)");
out->print_cr("");
out->print_cr(" redact-argument (Optional) Replace command-line arguments that match a");
out->print_cr(" semicolon-separated list of glob patterns, for example,");
out->print_cr(" *secret*;password*. Matching is case-insensitive, and the");
out->print_cr(" supported wildcards are '*' and '?'. To redact multiple arguments,");
out->print_cr(" use a literal space (' ') as a separator. For example, to match");
out->print_cr(" the two arguments --auth username:token, use the filter");
out->print_cr(" --auth *:*. Filters containing spaces must be quoted as a single");
out->print_cr(" command-line argument, for example,");
out->print_cr(" -XX:FlightRecorderOptions:'redact-argument=--auth *:*'.");
out->print_cr(" Arguments containing spaces might not be matched as expected.");
out->print_cr(" The option redact-argument is best-effort and applies only to");
out->print_cr(" command-line arguments in the jdk.JVMInformation event and to");
out->print_cr(" the java.command system property in the jdk.InitialSystemProperty");
out->print_cr(" event. Other events, such as jdk.ProcessStart (child processes),");
out->print_cr(" are not redacted.");
out->print_cr("");
out->print_cr(" If the redact-argument option is not specified, the following");
out->print_cr(" filters are used by default:");
out->print_cr("");
print_filters(out, JfrRedactedEvents::argument_filters());
out->print_cr("");
out->print_cr(" To load patterns from a file (one per line), use @<filename>.");
out->print_cr(" To add to the default patterns instead of replacing them, prefix");
out->print_cr(" the whole list with '+', for example, +*foo*;@redact.txt.");
out->print_cr(" Use 'none' (lowercase) to disable all redaction filters for");
out->print_cr(" command-line arguments. Redacted arguments will be replaced");
out->print_cr(" with '[REDACTED]'. (STRING, default filters)");
out->print_cr("");
out->print_cr(" redact-key (Optional) Replace the value of environment variables and system");
out->print_cr(" properties whose key matches a semicolon-separated list of glob");
out->print_cr(" patterns, for example, *password*;*token*. Matching is");
out->print_cr(" case-insensitive, and the supported wildcards are '*' and '?'.");
out->print_cr(" The option redact-key is best-effort and applies only to the");
out->print_cr(" jdk.InitialSystemProperty, jdk.InitialEnvironmentVariable and");
out->print_cr(" jdk.JVMInformation (-Dkey...) events. Other events, such as");
out->print_cr(" jdk.InitialSecurityProperty, are not redacted.");
out->print_cr("");
out->print_cr(" If the redact-key option is not specified, the");
out->print_cr(" following filters are used by default:");
out->print_cr("");
print_filters(out, JfrRedactedEvents::key_filters());
out->print_cr("");
out->print_cr(" To load patterns from a file (one per line), use @<filename>.");
out->print_cr(" To add to the default patterns instead of replacing them, prefix");
out->print_cr(" the whole list with '+', for example, +*cred*;@keys.txt.");
out->print_cr(" Use 'none' (lowercase) to disable all redaction filters for key");
out->print_cr(" matching. Redacted values will be replaced with '[REDACTED]'.");
out->print_cr(" (STRING, default filters)");
out->print_cr("");
out->print_cr("Options must be specified using the <key>=<value> syntax. Multiple options are separated");
out->print_cr("with a comma.");
out->print_cr("");
out->print_cr("Example usage:");
out->print_cr("");
out->print_cr(" -XX:FlightRecorderOptions:repository=/temporary,stackdepth=256");
out->print_cr(" -XX:FlightRecorderOptions:'redact-key=+*confidential*;*private*,redact-argument=+https://*:*@*'");
out->print_cr(" -XX:FlightRecorderOptions:'redact-argument=+-*private *'");
} else {
out->print_cr("Options must be specified using the <key> or <key>=<value> syntax.");
out->print_cr("");
out->print_cr("Example usage:");
out->print_cr("");
out->print_cr(" $ jcmd <pid> JFR.configure");
out->print_cr(" $ jcmd <pid> JFR.configure repositorypath=/temporary");
out->print_cr(" $ jcmd <pid> JFR.configure stackdepth=256");
out->print_cr(" $ jcmd <pid> JFR.configure memorysize=100M");
}
out->print_cr("");
}

3
src/hotspot/share/jfr/dcmd/jfrDcmds.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2012, 2024, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -202,6 +202,7 @@ class JfrConfigureFlightRecorderDCmd : public DCmdWithParser {
return "Low";
}
static int num_arguments() { return 10; }
static void print_help(outputStream* out, bool startup);
virtual void execute(DCmdSource source, TRAPS);
virtual void print_help(const char* name) const;
};

16
src/hotspot/share/jfr/metadata/metadata.xml
View File

@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Copyright (c) 2012, 2025, Oracle and/or its affiliates. All rights reserved.
Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
This code is free software; you can redistribute it and/or modify it
@ -500,11 +500,6 @@
<Field type="float" contentType="percentage" name="thresholdPercentage" label="Current IHOP Threshold" description="Current IHOP threshold in percent of old generation" />
<Field type="ulong" contentType="bytes" name="targetOccupancy" label="Target Occupancy" description="Target old generation occupancy to reach at the start of mixed GC" />
<Field type="ulong" contentType="bytes" name="currentOccupancy" label="Current Occupancy" description="Current old generation occupancy" />
<Field type="ulong" contentType="bytes" name="recentMutatorAllocationSize" label="Recent Mutator Allocation Size"
description="Mutator allocation during mutator operation in the most recent interval" />
<Field type="long" contentType="millis" name="recentMutatorDuration" label="Recent Mutator Duration" description="Time the mutator ran in the most recent interval" />
<Field type="double" contentType="bytes-per-second" name="recentAllocationRate" label="Recent Allocation Rate" description="Allocation rate of the mutator in the most recent interval in bytes/second" />
<Field type="long" contentType="millis" name="lastMarkingDuration" label="Last Marking Duration" description="Last time from the end of the last concurrent start to the first mixed GC" />
</Event>
<Event name="G1AdaptiveIHOP" category="Java Virtual Machine, GC, Detailed" label="G1 Adaptive IHOP Statistics" startTime="false"
@ -514,8 +509,15 @@
<Field type="float" contentType="percentage" name="thresholdPercentage" label="Threshold" description="Current IHOP threshold in percent of the internal target occupancy" />
<Field type="ulong" contentType="bytes" name="ihopTargetOccupancy" label="IHOP Target Occupancy" description="Internal target old generation occupancy to reach at the start of mixed GC" />
<Field type="ulong" contentType="bytes" name="currentOccupancy" label="Current Occupancy" description="Current old generation occupancy" />
<Field type="ulong" contentType="bytes" name="nonHumongousAllocation" label="Current Non-Humongous Allocation"
description="Current non-humongous allocation accumulated by the concurrent cycle tracker" />
<Field type="ulong" contentType="bytes" name="peakExtraHumongousOccupancy" label="Current Peak Extra Humongous Occupancy"
description="Current peak increase in humongous occupancy relative to the occupancy at concurrent cycle start" experimental="true" />
<Field type="ulong" contentType="bytes" name="additionalBufferSize" label="Additional Buffer" description="Additional buffer size" experimental="true" />
<Field type="double" contentType="bytes-per-second" name="predictedAllocationRate" label="Predicted Allocation Rate" description="Current predicted allocation rate for the mutator in bytes/second" />
<Field type="double" contentType="bytes-per-second" name="predictedNonHumongousAllocation" label="Predicted Non-Humongous Allocation Rate"
description="Predicted allocation rate of non-humongous objects into the old generation, in bytes per second" />
<Field type="ulong" contentType="bytes" name="predictedPeakExtraHumongousOccupancy" label="Predicted Peak Extra Humongous Occupancy"
description="Predicted peak increase in humongous occupancy during a concurrent cycle relative to the occupancy at concurrent cycle start" experimental="true" />
<Field type="long" contentType="millis" name="predictedMarkingDuration" label="Predicted Marking Duration"
description="Current predicted time from the end of the last concurrent start to the first mixed GC" />
<Field type="boolean" name="predictionActive" label="Prediction Active" description="Indicates whether the adaptive IHOP prediction is active" />

33
src/hotspot/share/jfr/periodic/jfrOSInterface.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2012, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -268,37 +268,6 @@ const char* JfrOSInterface::virtualization_name() {
return "No virtualization detected";
}
int JfrOSInterface::generate_initial_environment_variable_events() {
if (os::get_environ() == nullptr) {
return OS_ERR;
}
if (EventInitialEnvironmentVariable::is_enabled()) {
// One time stamp for all events, so they can be grouped together
JfrTicks time_stamp = JfrTicks::now();
for (char** p = os::get_environ(); *p != nullptr; p++) {
char* variable = *p;
char* equal_sign = strchr(variable, '=');
if (equal_sign != nullptr) {
// Extract key/value
ResourceMark rm;
ptrdiff_t key_length = equal_sign - variable;
char* key = NEW_RESOURCE_ARRAY(char, key_length + 1);
char* value = equal_sign + 1;
strncpy(key, variable, key_length);
key[key_length] = '\0';
EventInitialEnvironmentVariable event(UNTIMED);
event.set_starttime(time_stamp);
event.set_endtime(time_stamp);
event.set_key(key);
event.set_value(value);
event.commit();
}
}
}
return OS_OK;
}
int JfrOSInterface::system_processes(SystemProcess** sys_processes, int* no_of_sys_processes) {
return instance()._impl->system_processes(sys_processes, no_of_sys_processes);
}

3
src/hotspot/share/jfr/periodic/jfrOSInterface.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2012, 2019, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -53,7 +53,6 @@ class JfrOSInterface: public JfrCHeapObj {
static int cpu_loads_process(double* pjvmUserLoad, double* pjvmKernelLoad, double* psystemTotalLoad);
static int os_version(char** os_version);
static const char* virtualization_name();
static int generate_initial_environment_variable_events();
static int system_processes(SystemProcess** system_processes, int* no_of_sys_processes);
static int network_utilization(NetworkInterface** network_interfaces);
};

34
src/hotspot/share/jfr/periodic/jfrPeriodic.cpp
View File

@ -41,6 +41,7 @@
#include "jfr/periodic/jfrNativeMemoryEvent.hpp"
#include "jfr/periodic/jfrNetworkUtilization.hpp"
#include "jfr/periodic/jfrOSInterface.hpp"
#include "jfr/periodic/jfrRedactedEvents.hpp"
#include "jfr/periodic/jfrThreadCPULoadEvent.hpp"
#include "jfr/periodic/jfrThreadDumpEvent.hpp"
#include "jfr/recorder/jfrRecorder.hpp"
@ -100,17 +101,8 @@ TRACE_REQUEST_FUNC(ResidentSetSize) {
}
TRACE_REQUEST_FUNC(JVMInformation) {
ResourceMark rm;
EventJVMInformation event;
event.set_jvmName(VM_Version::vm_name());
event.set_jvmVersion(VM_Version::internal_vm_info_string());
event.set_javaArguments(Arguments::java_command());
event.set_jvmArguments(Arguments::jvm_args());
event.set_jvmFlags(Arguments::jvm_flags());
event.set_jvmStartTime(Management::vm_init_done_time());
event.set_pid(os::current_process_id());
event.commit();
}
JfrRedactedEvents::emit_jvm_information();
}
TRACE_REQUEST_FUNC(OSInformation) {
ResourceMark rm;
@ -169,7 +161,9 @@ TRACE_REQUEST_FUNC(NativeLibrary) {
}
TRACE_REQUEST_FUNC(InitialEnvironmentVariable) {
JfrOSInterface::generate_initial_environment_variable_events();
if (!JfrRedactedEvents::emit_initial_environment_variables()) {
log_debug(jfr, system)( "Unable to generate periodic event InitialEnvironmentVariable");
}
}
TRACE_REQUEST_FUNC(CPUInformation) {
@ -393,7 +387,7 @@ TRACE_REQUEST_FUNC(BooleanFlag) {
}
TRACE_REQUEST_FUNC(StringFlag) {
SEND_FLAGS_OF_TYPE(StringFlag, ccstr);
JfrRedactedEvents::emit_string_flags();
}
class VM_GC_SendObjectCountEvent : public VM_GC_HeapInspection {
@ -476,19 +470,7 @@ TRACE_REQUEST_FUNC(YoungGenerationConfiguration) {
}
TRACE_REQUEST_FUNC(InitialSystemProperty) {
SystemProperty* p = Arguments::system_properties();
JfrTicks time_stamp = JfrTicks::now();
while (p != nullptr) {
if (!p->internal()) {
EventInitialSystemProperty event(UNTIMED);
event.set_key(p->key());
event.set_value(p->value());
event.set_starttime(time_stamp);
event.set_endtime(time_stamp);
event.commit();
}
p = p->next();
}
JfrRedactedEvents::emit_initial_system_properties();
}
TRACE_REQUEST_FUNC(ThreadAllocationStatistics) {

663
src/hotspot/share/jfr/periodic/jfrRedactedEvents.cpp Normal file
View File

@ -0,0 +1,663 @@
/*
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*
*/
#include "jfr/jfrEvents.hpp"
#include "jfr/periodic/jfrRedactedEvents.hpp"
#include "jfr/utilities/jfrTime.hpp"
#include "logging/log.hpp"
#include "logging/logMessage.hpp"
#include "runtime/arguments.hpp"
#include "runtime/flags/jvmFlag.hpp"
#include "runtime/os.hpp"
#include "runtime/vm_version.hpp"
#include "services/diagnosticArgument.hpp"
#include "services/diagnosticFramework.hpp"
#include "services/management.hpp"
#include "utilities/ostream.hpp"
#include <ctype.h>
#include <stdio.h>
#include <string.h>
using StringArray = JfrRedactedEvents::StringArray;
using String = JfrRedactedEvents::String;
using StringFlag = JfrRedactedEvents::StringFlag;
using StringKeyValueArray = GrowableArray<JfrRedactedEvents::StringKeyValue*>*;
static const char REDACTED[] = "[REDACTED]";
static const char DELIMITER[] = " ";
static const char REDACT_ARGUMENT_EQUAL[] = "redact-argument=";
static const size_t REDACTED_LENGTH = sizeof(REDACTED) -1;
static const size_t DELIMITER_LENGTH = sizeof(DELIMITER) -1;
static const size_t REDACT_ARGUMENT_EQUAL_LENGTH = sizeof(REDACT_ARGUMENT_EQUAL) -1;
String* JfrRedactedEvents::_redacted_java_command_line = nullptr;
String* JfrRedactedEvents::_redacted_jvm_command_line = nullptr;
String* JfrRedactedEvents::_redacted_flags_command_line = nullptr;
String* JfrRedactedEvents::_redacted_flight_recorder_options = nullptr;
StringKeyValueArray JfrRedactedEvents::_initial_environment_variables = nullptr;
StringKeyValueArray JfrRedactedEvents::_initial_system_properties = nullptr;
GrowableArray<StringFlag*>* JfrRedactedEvents::_string_flags = nullptr;
StringArray* JfrRedactedEvents::_redacted_arguments = nullptr;
StringArray* JfrRedactedEvents::_argument_filters = nullptr;
StringArray* JfrRedactedEvents::_key_filters = nullptr;
bool JfrRedactedEvents::_initialized = false;
bool JfrRedactedEvents::set_argument_filter(const char* filters) {
assert (_argument_filters == nullptr, "invariant");
assert (filters != nullptr, "invariant");
_argument_filters = new StringArray();
return append_filters(_argument_filters, true, filters);
}
bool JfrRedactedEvents::set_key_filter(const char* filters) {
assert (_key_filters == nullptr, "invariant");
assert (filters != nullptr, "invariant");
_key_filters = new StringArray();
return append_filters(_key_filters, false, filters);
}
void JfrRedactedEvents::log_redaction() {
//At this point JFR is not enabled so events will not be emitted
ensure_initialized();
emit_initial_environment_variables(true);
emit_initial_system_properties(true);
emit_jvm_information(true);
emit_string_flags(true);
}
StringArray* JfrRedactedEvents::key_filters() {
ensure_initialized();
return _key_filters;
}
StringArray* JfrRedactedEvents::argument_filters() {
ensure_initialized();
return _argument_filters;
}
void JfrRedactedEvents::add_default_filters(StringArray* target, bool argument) {
if (argument) {
target->add("-*api*key *");
target->add("-*client*secret *");
target->add("-*credential *");
target->add("-*jaas*config *");
target->add("-*passphrase *");
target->add("-*passwd *");
target->add("-*password *");
target->add("-*private*key *");
target->add("-*pwd *");
target->add("-*secret *");
target->add("-*token *");
} else {
target->add("*auth*");
}
target->add("*api*key*");
target->add("*client*secret*");
target->add("*credential*");
target->add("*jaas*config*");
target->add("*passphrase*");
target->add("*passwd*");
target->add("*password*");
target->add("*private*key*");
target->add("*pwd*");
target->add("*secret*");
target->add("*token*");
}
bool JfrRedactedEvents::append_filters(StringArray* target, bool argument, const char* filters) {
const char* option_name = argument ? "redact-argument": "redact-key";
if (filters == nullptr) {
log_warning(jfr, redact)("Missing value for option -XX:FlightRecorderOptions:%s", option_name);
return false;
}
if (filters[0] == '\0') {
LogMessage(jfr, redact) msg;
msg.warning("Default redaction filters are replaced. Specify:");
msg.warning("-XX:FlightRecorderOptions:%s=none to disable filters without a warning.", option_name);
return true;
}
if (strcmp(filters, "none") == 0) {
return true;
}
if (filters[0] == '+') {
filters++;
add_default_filters(target, argument);
} else {
if (strncmp(filters, "none;", 5) != 0) {
LogMessage(jfr, redact) msg;
msg.warning("Default redaction filters are replaced. Prepend with '+' to add filters to the");
msg.warning("defaults, or specify -XX:FlightRecorderOptions:'%s=none;<filters>'", option_name);
msg.warning("to replace without a warning.");
} else {
filters += 5;
}
}
StringArray* filter_array = split(filters, ';');
bool success = true;
for (int i = 0; i < filter_array->length(); i++) {
const String* filter = filter_array->at(i);
if (strcmp(filter->text(), "none") == 0) {
log_error(jfr, redact)("'none' can only be used as the first filter.");
success = false;
break;
}
if (filter->at(0) == '@') {
const char* filename = filter->text() + 1;
if (!read_file(target, filename)) {
success = false;
}
} else {
target->add_contents(filter);
}
}
delete filter_array;
return success;
}
char* JfrRedactedEvents::new_redacted_text() {
char* result = NEW_C_HEAP_ARRAY(char, REDACTED_LENGTH + 1, mtTracing);
strncpy(result, REDACTED, REDACTED_LENGTH + 1);
return result;
}
bool JfrRedactedEvents::emit_initial_environment_variables(bool log) {
if (_initial_environment_variables == nullptr) {
ensure_initialized();
char** envp = os::get_environ();
if (envp == nullptr) {
return false;
}
_initial_environment_variables = make_array<StringKeyValue*>(0);
for (char** p = envp; *p != nullptr; ++p) {
char* variable = *p;
char* equal_sign = strchr(variable, '=');
if (equal_sign != nullptr) {
ptrdiff_t length = equal_sign - variable;
String* key = new String(variable, (size_t)length);
const char* value = equal_sign + 1;
if (is_redacted_key(key->text())) {
value = REDACTED;
if (log) {
log_debug(jfr, redact)("Redacted initial environment variable named '%s'", key->text());
}
}
_initial_environment_variables->append(new StringKeyValue(key, value));
}
}
}
JfrTicks time_stamp = JfrTicks::now();
for (int i = 0; i < _initial_environment_variables->length(); i++) {
StringKeyValue* pair = _initial_environment_variables->at(i);
EventInitialEnvironmentVariable event(UNTIMED);
event.set_starttime(time_stamp);
event.set_endtime(time_stamp);
event.set_key(pair->key());
event.set_value(pair->value());
event.commit();
}
return true;
}
void JfrRedactedEvents::emit_initial_system_properties(bool log) {
if (_initial_system_properties == nullptr) {
ensure_initialized();
_initial_system_properties = make_array<StringKeyValue*>(0);
for (SystemProperty* p = Arguments::system_properties(); p != nullptr; p = p->next()) {
if (!p->internal()) {
const char* value = p->value();
if (strcmp(p->key(), "sun.java.command") == 0) {
value = String::c_str(_redacted_java_command_line);
}
if (is_redacted_key(p->key())) {
value = REDACTED;
}
if (log && value != nullptr && strstr(value, REDACTED) != nullptr) {
log_debug(jfr, redact)("Redacted initial system property named '%s'", p->key());
}
_initial_system_properties->append(new StringKeyValue(p->key(), value));
}
}
}
JfrTicks time_stamp = JfrTicks::now();
for (int i = 0; i < _initial_system_properties->length(); i++) {
StringKeyValue* pair = _initial_system_properties->at(i);
EventInitialSystemProperty event(UNTIMED);
event.set_key(pair->key());
event.set_value(pair->value());
event.set_starttime(time_stamp);
event.set_endtime(time_stamp);
event.commit();
}
}
bool JfrRedactedEvents::match_flag(const char* flag_name, const char* arg) {
if (flag_name == nullptr || arg == nullptr) {
return false;
}
while (*flag_name) {
if (*arg != *flag_name) {
return false;
}
flag_name++;
arg++;
}
return *arg == '\0' || *arg == '=';
}
void JfrRedactedEvents::emit_string_flags(bool log) {
if (_string_flags == nullptr) {
ensure_initialized();
_string_flags = make_array<StringFlag*>(0);
JVMFlag *flag = JVMFlag::flags;
while (flag->name() != nullptr) {
if (flag->is_ccstr() && flag->is_unlocked()) {
const char* value = nullptr;
bool redacted = false;
for (int i = 0; i < _redacted_arguments->length(); i++) {
if (match_flag(flag->name(), _redacted_arguments->at(i)->text())) {
value = REDACTED;
redacted = true;
break;
}
}
if (strcmp("FlightRecorderOptions", flag->name()) == 0) {
if (_redacted_flight_recorder_options != nullptr) {
value = _redacted_flight_recorder_options->text();
redacted = true;
}
}
if (log && redacted) {
log_debug(jfr, redact)("Redacted string flag '%s'", flag->name());
}
_string_flags->append(new StringFlag(flag, value));
}
++flag;
}
}
for (int i = 0; i < _string_flags->length(); i++) {
StringFlag* sf = _string_flags->at(i);
const JVMFlag* flag = sf->jvm_flag();
EventStringFlag event;
event.set_name(flag->name());
if (sf->redacted_value() != nullptr) {
// If a flag is redacted and later changed at runtime,
// the event will still show the redacted text.
event.set_value(sf->redacted_value());
} else {
// If a flag is not redacted the first time,
// it will not be redacted later if it's changed at runtime.
event.set_value(flag->get_ccstr());
}
event.set_origin(static_cast<u8>(flag->get_origin()));
event.commit();
}
}
void JfrRedactedEvents::log_jvm_information_redaction(const char* type, const String* redacted) {
const char* text = String::c_str(redacted);
if (text != nullptr && strstr(text, REDACTED) != nullptr) {
log_debug(jfr, redact)("Redacted %s '%s'", type, text);
}
}
void JfrRedactedEvents::emit_jvm_information(bool log) {
ensure_initialized();
EventJVMInformation event;
event.set_jvmName(VM_Version::vm_name());
event.set_jvmVersion(VM_Version::internal_vm_info_string());
event.set_javaArguments(String::c_str(_redacted_java_command_line));
event.set_jvmArguments(String::c_str(_redacted_jvm_command_line));
event.set_jvmFlags(String::c_str(_redacted_flags_command_line));
event.set_jvmStartTime(Management::vm_init_done_time());
event.set_pid(os::current_process_id());
event.commit();
if (log) {
log_jvm_information_redaction("Java Arguments", _redacted_java_command_line);
log_jvm_information_redaction("JVM Arguments", _redacted_jvm_command_line);
log_jvm_information_redaction("JVM Flags", _redacted_flags_command_line);
}
}
void JfrRedactedEvents::ensure_initialized() {
if (_initialized) {
return;
}
if (_key_filters == nullptr) {
_key_filters = new StringArray();
add_default_filters(_key_filters, false);
}
if (_argument_filters == nullptr) {
_argument_filters = new StringArray();
add_default_filters(_argument_filters, true);
}
if (FlightRecorderOptions != nullptr) {
if (strstr(FlightRecorderOptions, REDACT_ARGUMENT_EQUAL) != nullptr) {
DCmdIter iterator(FlightRecorderOptions, ',');
stringStream result;
size_t pos = 0;
while(iterator.has_next()) {
CmdLine line = iterator.next();
const char* start = line.cmd_addr();
if (strncmp(start, REDACT_ARGUMENT_EQUAL, REDACT_ARGUMENT_EQUAL_LENGTH) == 0) {
result.print(REDACT_ARGUMENT_EQUAL);
result.print(REDACTED);
// Preserve ',' if there are more tokens
pos = iterator.has_next() ? iterator.cursor() - 1 : iterator.cursor();
}
while (pos < iterator.cursor()) {
result.write(FlightRecorderOptions + pos, 1);
pos++;
}
}
_redacted_flight_recorder_options = new String(result.base());
} else {
_redacted_flight_recorder_options = new String(FlightRecorderOptions);
}
}
_redacted_arguments = new StringArray();
StringArray* java_args = make_java_args_array();
_redacted_java_command_line = redact_command_line(java_args);
delete java_args;
StringArray* jvm_args = make_jvm_args_array(Arguments::jvm_args_array(), Arguments::num_jvm_args());
_redacted_jvm_command_line = redact_command_line(jvm_args);
delete jvm_args;
StringArray* flags_args = make_jvm_args_array(Arguments::jvm_flags_array(), Arguments::num_jvm_flags());
_redacted_flags_command_line = redact_command_line(flags_args);
delete flags_args;
_initialized = true;
}
String* JfrRedactedEvents::redact_command_line(StringArray* arguments) {
if (arguments == nullptr) {
return nullptr;
}
GrowableArray<StringArray*>* filters = make_array<StringArray*>(_argument_filters->length());
for (int i = 0 ; i < _argument_filters->length(); i++) {
filters->append(make_filter_array(_argument_filters->at(i)->text()));
}
StringArray* result = new StringArray();
int arg_index = 0;
while (arg_index < arguments->length()) {
int next_index = arg_index;
for (int i = 0; i < filters->length(); i++) {
next_index = match_arguments(filters->at(i), arguments, arg_index);
if (next_index > arg_index) {
break;
}
}
if (next_index > arg_index) {
for (int j = arg_index; j < next_index; j++) {
result->add(REDACTED);
const char* arg = arguments->at(j)->text();
if (arg != nullptr && strncmp(arg, "-XX:", 4) == 0) {
_redacted_arguments->add(arg + 4);
}
}
arg_index = next_index;
} else {
result->add_contents(arguments->at(arg_index));
arg_index++;
}
}
destroy_array(filters);
String* ret = concatenate(result);
delete result;
return ret;
}
String* JfrRedactedEvents::concatenate(StringArray* strings) {
size_t length = 0;
for (int i = 0; i < strings->length(); i++) {
length += strings->at(i)->length();
if (i != 0) {
length += 1;
}
}
String* text = new String(length);
size_t index = 0;
for (int i = 0; i < strings->length(); i++) {
if (i != 0) {
text->set(index++, ' ');
}
const String* s = strings->at(i);
size_t len = s->length();
for (size_t j = 0; j < len; j++) {
text->set(index++, s->at(j));
}
}
return text;
}
bool JfrRedactedEvents::equals_case_insensitive(char a, char b) {
return tolower((unsigned char)a) == tolower((unsigned char)b);
}
bool JfrRedactedEvents::is_redacted_key(const char* key) {
return match_key(_key_filters, key);
}
bool JfrRedactedEvents::is_separator(char c) {
return c == ' ' || c == '\t' || c == '\n' || c == '\r';
}
bool JfrRedactedEvents::is_whitespace(char c) {
return c == ' ';
}
StringArray* JfrRedactedEvents::make_filter_array(const char* filter) {
StringArray* result = new StringArray();
while (*filter) {
const char* q = strstr(filter, DELIMITER);
if (q == nullptr) {
size_t len = strlen(filter);
if (len > 0) {
result->add(filter, len);
}
break;
}
size_t len = (size_t)(q - filter);
if (len > 0) {
result->add(filter, len);
}
filter = q + DELIMITER_LENGTH;
}
return result;
}
StringArray* JfrRedactedEvents::make_jvm_args_array(char** jvm_args_array, int array_length) {
if (jvm_args_array == nullptr) {
return nullptr;
}
StringArray* result = new StringArray(array_length);
for(int i = 0; i < array_length; i++) {
char* argument = jvm_args_array[i];
if (_redacted_flight_recorder_options != nullptr &&
strncmp(argument, "-XX:FlightRecorderOptions", 25) == 0) {
const char* text = _redacted_flight_recorder_options->text();
size_t length = _redacted_flight_recorder_options->length();
// Length must be at least 26 or the JVM will not start.
result->add(new String(argument, 26, text, length));
continue;
}
if (strncmp(argument, "-D", 2) == 0) {
const char* key_start = argument + 2;
const char* eq = strchr(key_start, '=');
if (eq != nullptr) {
size_t key_length = (size_t)(eq - key_start);
String* key_tmp = new String(key_start, key_length);
bool redact = match_key(_key_filters, key_tmp->text());
delete key_tmp;
if (redact) {
size_t unsensitive_length = (size_t)(eq - argument) + 1;
result->add(new String(argument, unsensitive_length, REDACTED, REDACTED_LENGTH));
continue;
}
}
}
result->add(argument);
}
return result;
}
StringArray* JfrRedactedEvents::make_java_args_array() {
StringArray* array = new StringArray();
const char* p = Arguments::java_command();
if (p == nullptr) {
return array;
}
const char* end = p + strlen(p);
while (p < end) {
while (p < end && is_whitespace(*p)) { // Launcher separates arguments with ' '.
p++;
}
if (p >= end) {
break;
}
const char* start = p;
while (p < end && !is_whitespace(*p)) { // Launcher separates arguments with ' '.
p++;
}
size_t length = (size_t)(p - start);
if (length > 0) {
array->add(start, length);
}
}
return array;
}
bool JfrRedactedEvents::match(const char* filter, const char* text) {
const char* filter_last = nullptr;
const char* text_last = nullptr;
while (*text != '\0') {
if (*filter == '*') {
filter_last = filter;
text_last = text;
filter++;
} else if (*filter == '?' || equals_case_insensitive(*filter, *text)) {
filter++;
text++;
} else if (filter_last != nullptr) {
text_last++;
text = text_last;
filter = filter_last + 1;
} else {
return false;
}
}
while (*filter == '*') {
filter++;
}
return *filter == '\0';
}
int JfrRedactedEvents::match_arguments(StringArray* filter, StringArray* arguments, int arg_index) {
int index = arg_index;
if (arg_index + filter->length() <= arguments->length()) {
for (int i = 0; i < filter->length(); i++) {
if (!match(filter->at(i)->text(), arguments->at(index)->text())) {
return arg_index;
}
index++;
}
}
return index;
}
bool JfrRedactedEvents::match_key(StringArray* filters, const char* text) {
for (int i = 0; i < filters->length(); i++) {
const char* filter = filters->at(i)->text();
if (match(filter, text)) {
return true;
}
}
return false;
}
bool JfrRedactedEvents::read_file(StringArray* target, const char* filename) {
FILE* file = os::fopen(filename, "r");
if (file == nullptr) {
log_error(jfr, redact)("Failed to open redaction file: %s", filename);
return false;
}
stringStream ss;
while (true) {
const int c = fgetc(file);
if (c == EOF && ferror(file) != 0) {
log_error(jfr, redact)("Error reading redaction file: %s", filename);
fclose(file);
return false;
}
if (c == EOF || c == '\n') {
const char* line = ss.base();
size_t end = ss.size();
while (end > 0 && is_separator(line[end - 1])) {
end--;
}
if (end > 0) {
target->add(line, end);
}
ss.reset();
if (c == EOF) {
break;
}
} else {
ss.put((char)c);
}
}
fclose(file);
log_debug(jfr, redact)("Redaction file %s read successfully", filename);
return true;
}
StringArray* JfrRedactedEvents::split(const char* text, char separator) {
const char* last = text;
StringArray* result = new StringArray();
for (const char* position = text; *position != '\0'; position++) {
if (*position == separator) {
if (position > last) {
result->add(last, (size_t)(position - last));
}
last = position + 1;
}
}
if (*last != '\0') {
result->add(last, strlen(last));
}
return result;
}

223
src/hotspot/share/jfr/periodic/jfrRedactedEvents.hpp Normal file
View File

@ -0,0 +1,223 @@
/*
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 only, as
* published by the Free Software Foundation.
*
* This code is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* version 2 for more details (a copy is included in the LICENSE file that
* accompanied this code).
*
* You should have received a copy of the GNU General Public License version
* 2 along with this work; if not, write to the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
* or visit www.oracle.com if you need additional information or have any
* questions.
*
*/
#ifndef SHARE_JFR_PERIODIC_JFRREDACTEDEVENTS_HPP
#define SHARE_JFR_PERIODIC_JFRREDACTEDEVENTS_HPP
#include "memory/allocation.hpp"
#include "memory/allStatic.hpp"
#include "utilities/globalDefinitions.hpp"
#include "utilities/growableArray.hpp"
#include <string.h>
class JVMFlag;
class JfrRedactedEvents: public AllStatic {
public:
class StringArray;
// Called at startup
static void log_redaction();
static char* new_redacted_text();
static bool set_argument_filter(const char* filters);
static bool set_key_filter(const char* filters);
static StringArray* argument_filters();
static StringArray* key_filters();
// Synchronized in Java
static bool emit_initial_environment_variables(bool log = false);
static void emit_initial_system_properties(bool log = false);
static void emit_jvm_information(bool log = false);
static void emit_string_flags(bool log = false);
template<typename T> static GrowableArray<T>* make_array(int size) {
return new (mtTracing) GrowableArray<T>(size, mtTracing);
}
template<typename T> static void destroy_array(GrowableArray<T*>* array) {
if (array == nullptr) {
return;
}
for (int i = 0; i < array->length(); ++i) {
delete array->at(i);
}
delete array;
}
class String: public CHeapObj<mtTracing> {
public:
String(size_t length) : _length(length) {
_text = NEW_C_HEAP_ARRAY(char, length + 1, mtTracing);
memset(_text, 0, length + 1);
}
String(const char* source, size_t length) : String(length) {
memcpy(_text, source, length);
}
String(const char* source) : String(source, strlen(source)) {
}
String(const char* prefix, size_t prefix_length, const char* postfix, size_t postfix_length)
: String(prefix_length + postfix_length) {
memcpy(_text, prefix, prefix_length);
memcpy(_text + prefix_length, postfix, postfix_length);
}
~String() {
FREE_C_HEAP_ARRAY(_text);
}
const char* text() const {
return _text;
}
char at(size_t index) const {
assert(index < _length, "out of bounds");
return _text[index];
}
void set(size_t index, char c) {
assert(index < _length, "out of bounds");
_text[index] = c;
}
size_t length() const {
return _length;
}
static String* from_cstr(const char* s) {
return s == nullptr ? nullptr : new String(s);
}
static const char* c_str(const String* s) {
return s == nullptr ? nullptr : s->text();
}
private:
const size_t _length;
char* _text;
};
class StringArray: public CHeapObj<mtTracing> {
public:
StringArray() : _array(make_array<String*>(0)) {
}
StringArray(int capacity) : _array(make_array<String*>(capacity)) {
}
StringArray(const char* const array[], int count) : _array(make_array<String*>(count)) {
for (int i = 0; i < count; ++i) {
_array->append(new String(array[i]));
}
}
~StringArray() {
destroy_array(_array);
}
int length() const {
return _array->length();
}
const String* at(int i) const {
return _array->at(i);
}
void add(String* s) {
_array->append(s);
}
void add_contents(const String* s) {
add(s->text());
}
void add(const char* s) {
_array->append(new String(s));
}
void add(const char* s, size_t length) {
_array->append(new String(s, length));
}
private:
GrowableArray<String*>* const _array;
};
class StringKeyValue: public CHeapObj<mtTracing> {
public:
StringKeyValue(String* key, const char* value): _key(key), _value(String::from_cstr(value)) {
}
StringKeyValue(const char* key, const char* value) : StringKeyValue(new String(key), value) {
}
~StringKeyValue() {
delete _key;
delete _value;
}
const char* key() const {
return _key->text();
}
const char* value() const {
return String::c_str(_value);
}
private:
String* const _key;
String* const _value;
};
class StringFlag: public CHeapObj<mtTracing> {
public:
StringFlag(const JVMFlag* flag, const char* redacted_value)
: _flag(flag),
_redacted_value(String::from_cstr(redacted_value)) {
}
~StringFlag() {
delete _redacted_value;
}
const JVMFlag* jvm_flag() const {
return _flag;
}
const char* redacted_value() const {
return _redacted_value == nullptr ? nullptr : _redacted_value->text();
}
private:
const JVMFlag* const _flag;
String* _redacted_value;
};
private:
static bool _initialized;
static StringArray* _argument_filters;
static StringArray* _key_filters;
static StringArray* _redacted_arguments;
static String* _redacted_java_command_line;
static String* _redacted_jvm_command_line;
static String* _redacted_flags_command_line;
static String* _redacted_flight_recorder_options;
static GrowableArray<StringKeyValue*>* _initial_system_properties;
static GrowableArray<StringKeyValue*>* _initial_environment_variables;
static GrowableArray<StringFlag*>* _string_flags;
static bool append_filters(StringArray* target, bool argument, const char* filters);
static void add_default_filters(StringArray* target, bool argument);
static String* concatenate(StringArray* strings);
static bool equals_case_insensitive(char a, char b);
static bool is_redacted_key(const char* key);
static bool is_separator(char c);
static bool is_whitespace(char c);
static void ensure_initialized();
static StringArray* make_java_args_array();
static StringArray* make_jvm_args_array(char** jvm_args_array, int array_length);
static StringArray* make_filter_array(const char* filter);
static void log_jvm_information_redaction(const char* type, const String* redacted);
static bool match(const char* pattern, const char* text);
static bool match_flag(const char* flag_name, const char* argument);
static int match_arguments(StringArray* filter_array, StringArray* arguments, int arg_index);
static bool match_key(StringArray* array, const char* text);
static bool read_file(StringArray* target, const char* filename);
static String* redact_command_line(StringArray* arguments);
static StringArray* split(const char* text, char separator);
};
#endif // SHARE_JFR_PERIODIC_JFRREDACTEDEVENTS_HPP

4
src/hotspot/share/jfr/recorder/jfrRecorder.cpp
View File

@ -29,6 +29,7 @@
#include "jfr/jni/jfrJavaSupport.hpp"
#include "jfr/leakprofiler/sampling/objectSampler.hpp"
#include "jfr/periodic/jfrOSInterface.hpp"
#include "jfr/periodic/jfrRedactedEvents.hpp"
#include "jfr/periodic/sampling/jfrCPUTimeThreadSampler.hpp"
#include "jfr/periodic/sampling/jfrThreadSampler.hpp"
#include "jfr/recorder/checkpoint/jfrCheckpointManager.hpp"
@ -243,6 +244,9 @@ bool JfrRecorder::on_create_vm_2() {
}
bool JfrRecorder::on_create_vm_3() {
if (log_is_enabled(Debug, jfr, redact)) {
JfrRedactedEvents::log_redaction();
}
JVMTI_ONLY( assert(JvmtiEnvBase::get_phase() == JVMTI_PHASE_LIVE, "invalid init sequence, phase is %d", (int)JvmtiEnvBase::get_phase()); )
return CDSConfig::is_dumping_archive() || launch_command_line_recordings(JavaThread::current());
}

38
src/hotspot/share/jfr/recorder/service/jfrOptionSet.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2012, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2012, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -25,6 +25,7 @@
#include "cds/cdsConfig.hpp"
#include "classfile/javaClasses.hpp"
#include "jfr/dcmd/jfrDcmds.hpp"
#include "jfr/periodic/jfrRedactedEvents.hpp"
#include "jfr/recorder/service/jfrMemorySizer.hpp"
#include "jfr/recorder/service/jfrOptionSet.hpp"
#include "jfr/utilities/jfrAllocation.hpp"
@ -262,6 +263,20 @@ static DCmdArgument<bool> _dcmd_preserve_repository(
false,
default_preserve_repository);
static DCmdArgument<char*> _dcmd_redact_argument(
"redact-argument",
"Redact command line arguments",
"STRING",
false,
nullptr);
static DCmdArgument<char*> _dcmd_redact_key(
"redact-key",
"Redact environment variables and system properties",
"STRING",
false,
nullptr);
static DCmdParser _parser;
static void register_parser_options() {
@ -277,6 +292,8 @@ static void register_parser_options() {
_parser.add_dcmd_option(&_dcmd_retransform);
_parser.add_dcmd_option(&_dcmd_old_object_queue_size);
_parser.add_dcmd_option(&_dcmd_preserve_repository);
_parser.add_dcmd_option(&_dcmd_redact_argument);
_parser.add_dcmd_option(&_dcmd_redact_key);
DEBUG_ONLY(_parser.add_dcmd_option(&_dcmd_sample_protection);)
}
@ -285,6 +302,11 @@ static bool parse_flight_recorder_options_internal(TRAPS) {
return true;
}
const size_t length = strlen((const char*)FlightRecorderOptions);
if (strcmp(FlightRecorderOptions, "help") == 0) {
JfrConfigureFlightRecorderDCmd::print_help(tty, true);
vm_exit(0);
}
CmdLine cmdline((const char*)FlightRecorderOptions, length, true);
_parser.parse(&cmdline, ',', THREAD);
if (HAS_PENDING_EXCEPTION) {
@ -332,6 +354,20 @@ bool JfrOptionSet::initialize(JavaThread* thread) {
set_retransform(_dcmd_retransform.value());
}
set_old_object_queue_size(_dcmd_old_object_queue_size.value());
if (_dcmd_redact_argument.is_set()) {
if (!JfrRedactedEvents::set_argument_filter(_dcmd_redact_argument.value())) {
return false;
}
_dcmd_redact_argument.destroy_value();
_dcmd_redact_argument.set_value(JfrRedactedEvents::new_redacted_text());
}
if (_dcmd_redact_key.is_set()) {
if (!JfrRedactedEvents::set_key_filter(_dcmd_redact_key.value())) {
return false;
}
_dcmd_redact_key.destroy_value();
_dcmd_redact_key.set_value(JfrRedactedEvents::new_redacted_text());
}
return adjust_memory_options();
}

1
src/hotspot/share/logging/logTag.hpp
View File

@ -169,6 +169,7 @@ class outputStream;
LOG_TAG(ptrqueue) \
LOG_TAG(purge) \
LOG_TAG(record) \
LOG_TAG(redact) \
LOG_TAG(redefine) \
LOG_TAG(ref) \
LOG_TAG(refine) \

2
src/hotspot/share/nmt/virtualMemoryTracker.cpp
View File

@ -283,7 +283,7 @@ bool RegionIterator::next_committed(address& committed_start, size_t& committed_
const size_t page_sz = os::vm_page_size();
const size_t current_size = end() - _current_start;
if (os::committed_in_range(_current_start, current_size, committed_start, committed_size)) {
if (os::first_resident_in_range(_current_start, current_size, committed_start, committed_size)) {
assert(committed_start != nullptr, "Must be");
assert(committed_size > 0 && is_aligned(committed_size, os::vm_page_size()), "Must be");

2
src/hotspot/share/opto/c2compiler.cpp
View File

@ -819,6 +819,8 @@ bool C2Compiler::is_intrinsic_supported(vmIntrinsics::ID id) {
case vmIntrinsics::_poly1305_processBlocks:
case vmIntrinsics::_intpoly_montgomeryMult_P256:
case vmIntrinsics::_intpoly_assign:
case vmIntrinsics::_intpoly_mult_25519:
case vmIntrinsics::_intpoly_square_25519:
case vmIntrinsics::_updateCRC32:
case vmIntrinsics::_updateBytesCRC32:
case vmIntrinsics::_updateByteBufferCRC32:

2
src/hotspot/share/opto/escape.cpp
View File

@ -2272,6 +2272,8 @@ void ConnectionGraph::process_call_arguments(CallNode *call) {
strcmp(call->as_CallLeaf()->_name, "poly1305_processBlocks") == 0 ||
strcmp(call->as_CallLeaf()->_name, "intpoly_montgomeryMult_P256") == 0 ||
strcmp(call->as_CallLeaf()->_name, "intpoly_assign") == 0 ||
strcmp(call->as_CallLeaf()->_name, "intpoly_mult_25519") == 0 ||
strcmp(call->as_CallLeaf()->_name, "intpoly_square_25519") == 0 ||
strcmp(call->as_CallLeaf()->_name, "ghash_processBlocks") == 0 ||
strcmp(call->as_CallLeaf()->_name, "chacha20Block") == 0 ||
strcmp(call->as_CallLeaf()->_name, "kyberNtt") == 0 ||

68
src/hotspot/share/opto/library_call.cpp
View File

@ -666,6 +666,10 @@ bool LibraryCallKit::try_to_inline(int predicate) {
return inline_intpoly_montgomeryMult_P256();
case vmIntrinsics::_intpoly_assign:
return inline_intpoly_assign();
case vmIntrinsics::_intpoly_mult_25519:
return inline_intpoly_mult_25519();
case vmIntrinsics::_intpoly_square_25519:
return inline_intpoly_square_25519();
case vmIntrinsics::_encodeISOArray:
case vmIntrinsics::_encodeByteISOArray:
return inline_encodeISOArray(false);
@ -8373,6 +8377,70 @@ bool LibraryCallKit::inline_intpoly_assign() {
return true;
}
bool LibraryCallKit::inline_intpoly_mult_25519() {
address stubAddr;
const char *stubName;
assert(UseIntPoly25519Intrinsics, "need intpoly25519 intrinsics support");
assert(callee()->signature()->size() == 3, "intpoly_mult_25519 has %d parameters", callee()->signature()->size());
stubAddr = StubRoutines::intpoly_mult_25519();
stubName = "intpoly_mult_25519";
if (!stubAddr) return false;
null_check_receiver(); // null-check receiver
if (stopped()) return true;
Node* a = argument(1);
Node* b = argument(2);
Node* r = argument(3);
a = must_be_not_null(a, true);
b = must_be_not_null(b, true);
r = must_be_not_null(r, true);
Node* a_start = array_element_address(a, intcon(0), T_LONG);
assert(a_start, "a array is null");
Node* b_start = array_element_address(b, intcon(0), T_LONG);
assert(b_start, "b array is null");
Node* r_start = array_element_address(r, intcon(0), T_LONG);
assert(r_start, "r array is null");
Node* call = make_runtime_call(RC_LEAF | RC_NO_FP,
OptoRuntime::intpoly_mult_25519_Type(),
stubAddr, stubName, TypePtr::BOTTOM,
a_start, b_start, r_start);
return true;
}
bool LibraryCallKit::inline_intpoly_square_25519() {
address stubAddr;
const char *stubName;
assert(UseIntPoly25519Intrinsics, "need intpoly25519 intrinsics support");
assert(callee()->signature()->size() == 2, "intpoly_mult_25519 has %d parameters", callee()->signature()->size());
stubAddr = StubRoutines::intpoly_square_25519();
stubName = "intpoly_square_25519";
if (!stubAddr) return false;
null_check_receiver(); // null-check receiver
if (stopped()) return true;
Node* a = argument(1);
Node* r = argument(2);
a = must_be_not_null(a, true);
r = must_be_not_null(r, true);
Node* a_start = array_element_address(a, intcon(0), T_LONG);
assert(a_start, "a array is null");
Node* r_start = array_element_address(r, intcon(0), T_LONG);
assert(r_start, "r array is null");
Node* call = make_runtime_call(RC_LEAF | RC_NO_FP,
OptoRuntime::intpoly_square_25519_Type(),
stubAddr, stubName, TypePtr::BOTTOM,
a_start, r_start);
return true;
}
//------------------------------inline_digestBase_implCompress-----------------------
//
// Calculate MD5 for single-block byte[] array.

2
src/hotspot/share/opto/library_call.hpp
View File

@ -343,6 +343,8 @@ class LibraryCallKit : public GraphKit {
bool inline_poly1305_processBlocks();
bool inline_intpoly_montgomeryMult_P256();
bool inline_intpoly_assign();
bool inline_intpoly_mult_25519();
bool inline_intpoly_square_25519();
bool inline_digestBase_implCompress(vmIntrinsics::ID id);
bool inline_keccak(vmIntrinsics::ID id);
bool inline_digestBase_implCompressMB(int predicate);

41
src/hotspot/share/opto/runtime.cpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 1998, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 1998, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -237,6 +237,8 @@ const TypeFunc* OptoRuntime::_string_IndexOf_Type = nullptr;
const TypeFunc* OptoRuntime::_poly1305_processBlocks_Type = nullptr;
const TypeFunc* OptoRuntime::_intpoly_montgomeryMult_P256_Type = nullptr;
const TypeFunc* OptoRuntime::_intpoly_assign_Type = nullptr;
const TypeFunc* OptoRuntime::_intpoly_mult_25519_Type = nullptr;
const TypeFunc* OptoRuntime::_intpoly_square_25519_Type = nullptr;
const TypeFunc* OptoRuntime::_updateBytesCRC32_Type = nullptr;
const TypeFunc* OptoRuntime::_updateBytesCRC32C_Type = nullptr;
const TypeFunc* OptoRuntime::_updateBytesAdler32_Type = nullptr;
@ -1786,6 +1788,41 @@ static const TypeFunc* make_intpoly_assign_Type() {
return TypeFunc::make(domain, range);
}
static const TypeFunc* make_intpoly_mult_25519_Type() {
int argcnt = 3;
const Type** fields = TypeTuple::fields(argcnt);
int argp = TypeFunc::Parms;
fields[argp++] = TypePtr::NOTNULL; // a array
fields[argp++] = TypePtr::NOTNULL; // b array
fields[argp++] = TypePtr::NOTNULL; // r(esult) array
assert(argp == TypeFunc::Parms + argcnt, "correct decoding");
const TypeTuple* domain = TypeTuple::make(TypeFunc::Parms+argcnt, fields);
// result type needed
fields = TypeTuple::fields(1);
fields[TypeFunc::Parms + 0] = nullptr; // void
const TypeTuple* range = TypeTuple::make(TypeFunc::Parms, fields);
return TypeFunc::make(domain, range);
}
static const TypeFunc* make_intpoly_square_25519_Type() {
int argcnt = 2;
const Type** fields = TypeTuple::fields(argcnt);
int argp = TypeFunc::Parms;
fields[argp++] = TypePtr::NOTNULL; // a array
fields[argp++] = TypePtr::NOTNULL; // r(esult) array
assert(argp == TypeFunc::Parms + argcnt, "correct decoding");
const TypeTuple* domain = TypeTuple::make(TypeFunc::Parms+argcnt, fields);
// result type needed
fields = TypeTuple::fields(1);
fields[TypeFunc::Parms + 0] = nullptr; // void
const TypeTuple* range = TypeTuple::make(TypeFunc::Parms, fields);
return TypeFunc::make(domain, range);
}
//------------- Interpreter state for on stack replacement
static const TypeFunc* make_osr_end_Type() {
// create input type (domain)
@ -2354,6 +2391,8 @@ void OptoRuntime::initialize_types() {
_poly1305_processBlocks_Type = make_poly1305_processBlocks_Type();
_intpoly_montgomeryMult_P256_Type = make_intpoly_montgomeryMult_P256_Type();
_intpoly_assign_Type = make_intpoly_assign_Type();
_intpoly_mult_25519_Type = make_intpoly_mult_25519_Type();
_intpoly_square_25519_Type = make_intpoly_square_25519_Type();
_updateBytesCRC32_Type = make_updateBytesCRC32_Type();
_updateBytesCRC32C_Type = make_updateBytesCRC32C_Type();
_updateBytesAdler32_Type = make_updateBytesAdler32_Type();

16
src/hotspot/share/opto/runtime.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 1998, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 1998, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -190,6 +190,8 @@ class OptoRuntime : public AllStatic {
static const TypeFunc* _poly1305_processBlocks_Type;
static const TypeFunc* _intpoly_montgomeryMult_P256_Type;
static const TypeFunc* _intpoly_assign_Type;
static const TypeFunc* _intpoly_mult_25519_Type;
static const TypeFunc* _intpoly_square_25519_Type;
static const TypeFunc* _updateBytesCRC32_Type;
static const TypeFunc* _updateBytesCRC32C_Type;
static const TypeFunc* _updateBytesAdler32_Type;
@ -687,6 +689,18 @@ private:
return _intpoly_assign_Type;
}
// IntegerPolynomial25519 multiply function
static inline const TypeFunc* intpoly_mult_25519_Type() {
assert(_intpoly_mult_25519_Type != nullptr, "should be initialized");
return _intpoly_mult_25519_Type;
}
// IntegerPolynomial25519 square function
static inline const TypeFunc* intpoly_square_25519_Type() {
assert(_intpoly_square_25519_Type != nullptr, "should be initialized");
return _intpoly_square_25519_Type;
}
/**
* int updateBytesCRC32(int crc, byte* b, int len)
*/

4
src/hotspot/share/opto/vectorization.cpp
View File

@ -1744,8 +1744,8 @@ AlignmentSolution* AlignmentSolver::solve() const {
// And since abs(C_pre) < aw, the solutions of (4a, b, c) can now only be constrained or empty.
// But since we already handled the empty case, the solutions are now all constrained.
assert(eq4a_state == EQ4::State::CONSTRAINED &&
eq4a_state == EQ4::State::CONSTRAINED &&
eq4a_state == EQ4::State::CONSTRAINED, "all must be constrained now");
eq4b_state == EQ4::State::CONSTRAINED &&
eq4c_state == EQ4::State::CONSTRAINED, "all must be constrained now");
// And since they are all constrained, we must have:
//

4
src/hotspot/share/prims/jvmtiExport.cpp
View File

@ -2269,6 +2269,8 @@ void JvmtiExport::post_field_access_by_jni(JavaThread *thread, oop obj,
RegisterMap::ProcessFrames::skip,
RegisterMap::WalkContinuation::skip);
javaVFrame *jvf = thread->last_java_vframe(&reg_map);
assert(jvf != nullptr, "last frame shouldn't be null");
Method* method = jvf->method();
address address = jvf->method()->code_base();
@ -2367,6 +2369,8 @@ void JvmtiExport::post_field_modification_by_jni(JavaThread *thread, oop obj,
RegisterMap::ProcessFrames::skip,
RegisterMap::WalkContinuation::skip);
javaVFrame *jvf = thread->last_java_vframe(&reg_map);
assert(jvf != nullptr, "last frame shouldn't be null");
Method* method = jvf->method();
address address = jvf->method()->code_base();

10
src/hotspot/share/runtime/continuation.cpp
View File

@ -153,12 +153,6 @@ freeze_result Continuation::try_preempt(JavaThread* current, oop continuation) {
return res;
}
#ifndef PRODUCT
static jlong java_tid(JavaThread* thread) {
return java_lang_Thread::thread_id(thread->threadObj());
}
#endif
ContinuationEntry* Continuation::get_continuation_entry_for_continuation(JavaThread* thread, oop continuation) {
if (thread == nullptr || continuation == nullptr) {
return nullptr;
@ -388,9 +382,9 @@ frame Continuation::continuation_bottom_sender(JavaThread* thread, const frame&
ContinuationEntry* ce = get_continuation_entry_for_sp(thread, callee.sp());
assert(ce != nullptr, "callee.sp(): " INTPTR_FORMAT, p2i(callee.sp()));
log_develop_debug(continuations)("continuation_bottom_sender: [" JLONG_FORMAT "] [%d] callee: " INTPTR_FORMAT
log_develop_debug(continuations)("continuation_bottom_sender: [" UINT64_FORMAT "] [%d] callee: " INTPTR_FORMAT
" sender_sp: " INTPTR_FORMAT,
java_tid(thread), thread->osthread()->thread_id(), p2i(callee.sp()), p2i(sender_sp));
thread->monitor_owner_id(), thread->osthread()->thread_id(), p2i(callee.sp()), p2i(sender_sp));
frame entry = ce->to_frame();
if (callee.is_interpreted_frame()) {

6
src/hotspot/share/runtime/globals.hpp
View File

@ -229,9 +229,13 @@ const int ObjectAlignmentInBytes = 8;
\
product(bool, UsePoly1305Intrinsics, false, DIAGNOSTIC, \
"Use intrinsics for sun.security.util.math.intpoly") \
product(bool, UseIntPolyIntrinsics, false, DIAGNOSTIC, \
\
product(bool, UseIntPolyIntrinsics, false, DIAGNOSTIC, \
"Use intrinsics for sun.security.util.math.intpoly.MontgomeryIntegerPolynomialP256") \
\
product(bool, UseIntPoly25519Intrinsics, false, DIAGNOSTIC, \
"Use intrinsics for sun.security.util.math.intpoly.IntegerPolynomial25519") \
\
product(size_t, LargePageSizeInBytes, 0, \
"Maximum large page size used (0 will use the default large " \
"page size for the environment as the maximum) " \

6
src/hotspot/share/runtime/os.hpp
View File

@ -440,9 +440,9 @@ class os: AllStatic {
public:
// get allowed minimum java stack size
static jlong get_minimum_java_stack_size();
// Find committed memory region within specified range (start, start + size),
// return true if found any
static bool committed_in_range(address start, size_t size, address& committed_start, size_t& committed_size);
// Find the first resident memory region within the specified range (start, start + size) beginning at the start address.
// Returns true if successful or false if none are found.
static bool first_resident_in_range(address start, size_t size, address& resident_start, size_t& resident_size);
// OS interface to Virtual Memory

8
src/hotspot/share/runtime/stubDeclarations.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2026, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2025, Red Hat, Inc. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
@ -801,6 +801,12 @@
intpoly_montgomeryMult_P256, intpoly_montgomeryMult_P256) \
do_stub(compiler, intpoly_assign) \
do_entry(compiler, intpoly_assign, intpoly_assign, intpoly_assign) \
do_stub(compiler, intpoly_mult_25519) \
do_entry(compiler, intpoly_mult_25519, \
intpoly_mult_25519, intpoly_mult_25519) \
do_stub(compiler, intpoly_square_25519) \
do_entry(compiler, intpoly_square_25519, \
intpoly_square_25519, intpoly_square_25519) \
do_stub(compiler, md5_implCompress) \
do_entry(compiler, md5_implCompress, md5_implCompress, \
md5_implCompress) \

5
src/hotspot/share/services/diagnosticFramework.hpp
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2011, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2011, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -85,6 +85,9 @@ public:
// instance to the caller.
return line;
}
size_t cursor() const {
return _cursor;
}
};
// Iterator class to iterate over diagnostic command arguments

12
src/java.base/share/classes/java/lang/classfile/ClassFile.java
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2022, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2022, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -1046,6 +1046,14 @@ public sealed interface ClassFile
*/
int JAVA_27_VERSION = 71;
/**
* The class major version introduced by Java SE 28, {@value}.
*
* @see ClassFileFormatVersion#RELEASE_28
* @since 28
*/
int JAVA_28_VERSION = 72;
/**
* A minor version number {@value} indicating a class uses preview features
* of a Java SE release since 12, for major versions {@value
@ -1057,7 +1065,7 @@ public sealed interface ClassFile
* {@return the latest class major version supported by the current runtime}
*/
static int latestMajorVersion() {
return JAVA_27_VERSION;
return JAVA_28_VERSION;
}
/**

16
src/java.base/share/classes/java/lang/reflect/ClassFileFormatVersion.java
View File

@ -1,5 +1,5 @@
/*
* Copyright (c) 2022, 2025, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2022, 2026, Oracle and/or its affiliates. All rights reserved.
* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
*
* This code is free software; you can redistribute it and/or modify it
@ -395,6 +395,18 @@ public enum ClassFileFormatVersion {
* <cite>The Java Virtual Machine Specification, Java SE 27 Edition</cite></a>
*/
RELEASE_27(71),
/**
* The version introduced by the Java Platform, Standard Edition
* 28.
*
* @since 28
*
* @see <a
* href="https://docs.oracle.com/en/java/javase/28/docs/specs/jvms/index.html">
* <cite>The Java Virtual Machine Specification, Java SE 28 Edition</cite></a>
*/
RELEASE_28(72),
; // Reduce code churn when appending new constants
// Note to maintainers: when adding constants for newer releases,
@ -410,7 +422,7 @@ public enum ClassFileFormatVersion {
* {@return the latest class file format version}
*/
public static ClassFileFormatVersion latest() {
return RELEASE_27;
return RELEASE_28;
}
/**

Some files were not shown because too many files have changed in this diff Show More