Use cpp11 by mgirlich · Pull Request #405 · r-lib/xml2

mgirlich · 2023-09-04T08:59:37Z

how to deal with memcpy()?
is the performance hit of the string handling relevant?
xml2_node.cpp can be tackled after Replace S3 dispatch by C implementation #400 is merged.

mgirlich · 2023-09-04T09:13:40Z

src/xml2_url.cpp

-    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(STRING_ELT(x_sxp, i));
-    SET_STRING_ELT(out, i, Xml2String(xmlBuildURI(uri, base_uri)).asRString());
+    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(x_sxp[i]);
+    out[i] = Xml2String(xmlBuildURI(uri, base_uri)).asRString();


I guess the slow down due to cpp11 protection probably doesn't matter so much here (https://cpp11.r-lib.org/articles/FAQ.html#ok-but-i-really-want-to-call-cpp11unwind_protect-manually).

Yea if it isn't critical then I'd probably leave as is

mgirlich · 2023-09-04T09:15:14Z

src/xml2_url.cpp

  for (int i = 0; i < n; ++i) {
-    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(STRING_ELT(x_sxp, i));
-    SET_STRING_ELT(out, i, Xml2String(xmlBuildURI(uri, base_uri)).asRString());
+    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(x_sxp[i]);


Probably we can somehow leave away the Rf_translateCharUTF8() due to using cpp11.

Hmm, I don't think cpp11 attempts to convert to UTF8 in this case

When x_sxp is created as cpp11::strings, it just wraps the character vector SEXP with no translation on the elements

When x_sxp[i] runs, it extracts the CHARSXP and turns it into a r_string, but that just calls this very simple constructor https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L17C10-L17C10

On the way into Rf_translateCharUTF8() the r_string is automatically changed back to a SEXP by this operator, which also doesn't translate https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L22

It looks like if we went through an intermediate std::string it would translate, but what you currently have seems ok
https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L29

mgirlich · 2023-09-04T09:17:32Z

src/xml2_url.cpp

-  if (Rf_xlength(base_sxp) > 1) {
-    Rf_error("Base URL must be length 1");
+const xmlChar* to_xml_chr(cpp11::strings x, const char* arg) {
+  if (x.size() > 1) {


Might make more sense to check this in R.

mgirlich · 2023-09-04T11:51:06Z

src/connection.h

 SEXP read_bin(SEXP con, size_t bytes = 64 * 1024);
-SEXP write_bin(SEXP data, SEXP con);
-
-inline SEXP R_GetConnection(SEXP con) { return con; }


I removed R_GetConnection() as it seemed unnecessary.

mgirlich · 2023-09-04T11:51:20Z

src/connection.h

-
-inline SEXP R_GetConnection(SEXP con) { return con; }
-
-inline size_t R_ReadConnection(SEXP con, void* buf, size_t n) {


R_ReadConnection() wasn't used anywhere.

mgirlich · 2023-09-04T11:54:00Z

src/connection.h

-}
+cpp11::sexp write_bin(cpp11::sexp data, cpp11::sexp con);

 inline size_t R_WriteConnection(SEXP con, void* buf, size_t n) {


This uses memcpy() and I wasn't sure how to do this properly with cpp11.

I don't think there is an alternative to memcpy(), but you could allocate a cpp11 raws and then do payload.sexp() to get access to the SEXP you can call RAW() on. I don't think that is too awful, and at least lets cpp11 manage the memory

DavisVaughan · 2023-10-31T14:16:07Z

src/connection.h

-}
+cpp11::sexp write_bin(cpp11::sexp data, cpp11::sexp con);

 inline size_t R_WriteConnection(SEXP con, void* buf, size_t n) {


I don't think there is an alternative to memcpy(), but you could allocate a cpp11 raws and then do payload.sexp() to get access to the SEXP you can call RAW() on. I don't think that is too awful, and at least lets cpp11 manage the memory

DavisVaughan · 2023-10-31T14:21:10Z

src/xml2_namespace.cpp

-extern "C" SEXP unique_ns(SEXP ns) {
+[[cpp11::register]]
+cpp11::sexp unique_ns(SEXP ns) {
  BEGIN_CPP


And in the other places below. Do we need this macro at all anymore? I assume if you go through cpp11 everywhere it will wrap it in try/catch for you

@DavisVaughan If you could merge #400 then I could also finish xml2_node.cpp and I think then it should be safe to remove the macros.

src/xml2_output.cpp

src/xml2_url.cpp

DavisVaughan · 2023-10-31T14:36:03Z

src/xml2_url.cpp

  for (int i = 0; i < n; ++i) {
-    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(STRING_ELT(x_sxp, i));
-    SET_STRING_ELT(out, i, Xml2String(xmlBuildURI(uri, base_uri)).asRString());
+    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(x_sxp[i]);


Hmm, I don't think cpp11 attempts to convert to UTF8 in this case

When x_sxp is created as cpp11::strings, it just wraps the character vector SEXP with no translation on the elements

When x_sxp[i] runs, it extracts the CHARSXP and turns it into a r_string, but that just calls this very simple constructor https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L17C10-L17C10

On the way into Rf_translateCharUTF8() the r_string is automatically changed back to a SEXP by this operator, which also doesn't translate https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L22

It looks like if we went through an intermediate std::string it would translate, but what you currently have seems ok
https://github.com/r-lib/cpp11/blob/500f642b4ea132ec8c168fc70a28e81e9510ece3/inst/include/cpp11/r_string.hpp#L29

DavisVaughan · 2023-10-31T14:37:50Z

src/xml2_url.cpp

-    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(STRING_ELT(x_sxp, i));
-    SET_STRING_ELT(out, i, Xml2String(xmlBuildURI(uri, base_uri)).asRString());
+    const xmlChar* uri = (xmlChar*) Rf_translateCharUTF8(x_sxp[i]);
+    out[i] = Xml2String(xmlBuildURI(uri, base_uri)).asRString();


Yea if it isn't critical then I'd probably leave as is

src/xml2_url.cpp

DavisVaughan · 2023-10-31T14:44:22Z

src/xml2_url.cpp

-    SET_STRING_ELT(scheme, i, Rf_mkChar(uri->scheme == NULL ? "" : uri->scheme));
-    SET_STRING_ELT(server, i, Rf_mkChar(uri->server == NULL ? "" : uri->server));
-    INTEGER(port)[i] = uri->port == 0 ? NA_INTEGER : uri->port;
-    SET_STRING_ELT(user, i, Rf_mkChar(uri->user == NULL ? "" : uri->user));
-    SET_STRING_ELT(path, i, Rf_mkChar(uri->path  == NULL ? "" : uri->path));
-    SET_STRING_ELT(fragment, i, Rf_mkChar(uri->fragment == NULL ? "" : uri->fragment));
+    scheme[i] = uri->scheme == NULL ? "" : uri->scheme;
+    server[i] = uri->server == NULL ? "" : uri->server;
+    port[i] = uri->port == 0 ? NA_INTEGER : uri->port;
+    user[i] = uri->user == NULL ? "" : uri->user;
+    path[i] = uri->path  == NULL ? "" : uri->path;
+    fragment[i] = uri->fragment == NULL ? "" : uri->fragment;


This one might get significantly slower, but again it doesn't seem like it is used all that often? I couldn't really say

I also think it shouldn't be used that often.

#Conflicts: # R/xml2-package.R # src/init.c

mgirlich · 2023-11-03T10:10:10Z

@DavisVaughan I now also migrated xml2_node.cpp to cpp11. But the performance hit from string handling really bites us now. It is nearly as slow as before replacing the S3 dispatch.

DavisVaughan · 2023-11-03T12:08:10Z

Is it one or two functions in particular?

mgirlich · 2023-11-03T12:41:30Z

I haven't looked into this in more detail but only checked xml_name() (see the benchmark in #400). The easiest would probably be to check and carefully revert some of the changes and then wrap things with cpp11::unwind_protect().

mgirlich added 3 commits September 4, 2023 07:51

use_cpp11()

5b11a52

Use cpp11 register mechanism

1fd06ac

Migrate xml2_url.cpp

48ab979

mgirlich force-pushed the use-cpp11 branch from 689a2f7 to 48ab979 Compare September 4, 2023 09:06

Replace extern "C" by cpp11::sexp

992cd01

mgirlich commented Sep 4, 2023

View reviewed changes

mgirlich added 4 commits September 4, 2023 11:10

Remove unused R_ReadConnection()

44eddbe

Remove unnecessary R_GetConnection()

66f4c60

Migrate xml2_output.cpp

cd54aed

Convert xml2_doc.cpp

e351747

mgirlich commented Sep 4, 2023

View reviewed changes

mgirlich added 4 commits September 4, 2023 12:23

Convert xml2_schema.cpp

60ae697

Convert xml2_path.cpp

ae3a6ec

Fix compilation issues

33d6c7e

Minor things

8029c30

hadley requested a review from DavisVaughan October 30, 2023 18:30

DavisVaughan approved these changes Oct 31, 2023

View reviewed changes

mgirlich added 9 commits November 2, 2023 05:57

As as_cpp<int>()

33561d8

Use R_xlen_t for size() method

10f3f75

Use cpp11 raws with memcpy

d43f4bf

Check for empty vectors as well

ea377a9

Merge commit '005b78659c2271a03ed016f13df0d32f8b1e66e5'

51cbbaf

#Conflicts: # R/xml2-package.R # src/init.c

Fix ambiguous initializer

31e7555

More cpp11 i xml2_xpath.cpp

b7ff753

Minor changes

50489ac

Use cpp11 in xml2_utils.h

6f0e353

mgirlich force-pushed the use-cpp11 branch from e09d20c to 6f0e353 Compare November 2, 2023 12:37

mgirlich added 6 commits November 2, 2023 15:21

Merge commit 'e088a1c892e97436fdd3263c6cfc6df61e47a365'

0352dfa

Use cpp11 in xml2_node.cpp

2539195

Fix compilation issues

b65e011

Use cpp11 in namespace and connection

f73bb41

Remove cpp wrapper macro

2e85921

Some more cleanup

01c41e7

jeroen force-pushed the main branch 2 times, most recently from 8f62602 to 9c3fc65 Compare December 16, 2024 00:06

jeroen force-pushed the main branch 3 times, most recently from 463c7e9 to ece90bb Compare March 1, 2025 19:42

jeroen force-pushed the main branch 2 times, most recently from 88024a3 to f9c5fe3 Compare August 20, 2025 10:52

jeroen force-pushed the main branch from eee310d to 39e3da6 Compare January 17, 2026 12:36


		inline SEXP R_GetConnection(SEXP con) { return con; }

		inline size_t R_ReadConnection(SEXP con, void* buf, size_t n) {

Conversation

mgirlich commented Sep 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgirlich Sep 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mgirlich commented Nov 3, 2023

Uh oh!

DavisVaughan commented Nov 3, 2023

Uh oh!

mgirlich commented Nov 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mgirlich commented Sep 4, 2023 •

edited

Loading

mgirlich Sep 4, 2023 •

edited

Loading