Vm-dev December 2018

vm-dev@lists.squeakfoundation.org

26 participants
279 discussions

[OpenSmalltalk/opensmalltalk-vm] 1f6163: Handle UTF16 logName and serviceName
by GitHub 31 Dec '18

31 Dec '18

Branch: refs/heads/WIN64_UNICODE Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 1f616371ff559224ad6200d1a1109e6c1a5007b3 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/1f616371ff559224ad… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32Service.c Log Message: ----------- Handle UTF16 logName and serviceName These are the last two obstacles that generate compiler warnings with -DUNICODE No compiler warning does not mean that UNICODE is OK and ready to go. It means that we have at least eliminated all the trivial TCHAR*/char*/WCHAR* mismatch (and there were a bunch of them!!!) Also the code is like a battlefield with lot of different ad hoc recipes, that would deserve more uniform approach (refactoring) But small steps! To reach the current stage, we have to overhaul printCommandLine() It can't answer a TCHAR* when sqMain expects a char*argv[]... So, like what is done in WinMain thru getCommandLineW, we first produce a Wide command line, then convert to UTF8. There is currently no provision for conversion failure: I don't know how to report and exit when the VM is ran as a service. Note that command line parsing is then working the other way around: some of the arguments are converted back to UTF16. I call this style tricotage coding: une maille à l'endroit, une maille à l'envers. Maybe a deeper change will be to WCHAR* all the way down, but let's differ this decision. Small steps! Note that another source of problems is RegQueryValueExW. When we query a WCHAR* string, it is not necessarily NULL terminated. But every answer is handled as raw un-interpretd-bytes (char *), whose byte length is answered in dwSize. That's not the WCHAR character length! So buffer[dwSize] = 0 is not the right thing: - if buffer is declared WVCHAR*, this would write the NULL well beyond the real string length (and eventually overrun the buffer); - if buffer is declared char*, this would not guaranty a terminating NULL WCHAR (we need 2 zero bytes to make a NULL WCHAR). Usage of this function will require a careful review, the price of low level API... Also note that we seem to compile with flag -DNO_SERVICE right now. Why? I have not enough knowledge in this area and lack tests/examples. In that conditions, it's not easy to test the modifications! Any help will be greatly appreciated. The good part is that It won't break current VM usage... You know what I think of dead-code, but half alive Frankenstein code scares me as well ;) **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[OpenSmalltalk/opensmalltalk-vm] f25041: We can't compare a TCHAR*windowClassName with a ch...
by GitHub 31 Dec '18

31 Dec '18

Branch: refs/heads/WIN64_UNICODE Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: f25041584cb703d00ef9677b4403c50244daa024 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/f25041584cb703d00e… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- We can't compare a TCHAR*windowClassName with a char*buf Even if buf were re-interpreted as WCHAR* when -DUNICODE, strcmp wouldn't do the right thing (it will stop at first ASCII because high 8 bits will be zero!) We thus use the TCHAR*dedicated `_tcscmp`. If ever we want to switch to UTF16 (WCHAR*) windowClassName, then it will be `wcscmp`. Commit: 84a8d172d1c524487cf9c2e06f2fc807ef27c6d9 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/84a8d172d1c524487c… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Prefs.c Log Message: ----------- Tu quoque NewspeakVM, frustra TEXT macro... That's the limit of using compiler warnings: we only focus on the sections we compile... BTW, ifdef NewspeakVM, OK, but what do Pharo people think about it? Commit: 7d3264e523709ea92aa1a70d1c9a97a863d3c0a8 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/7d3264e523709ea92a… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/plugins/SocketPlugin/sqWin32NewNet.c Log Message: ----------- gai_strerror returns a TCHAR*, we cannot simply fprintf it... Choose the UNICODE variant, because error messages are presumably localized an may use non ASCII characters The alternative would be to use `_ftprintf(stderr,TEXT("%s"),gai_strerror(gaiError))` and let -DUNICODE decide... Commit: 6ff9625b6cb47487bf5421921c9b7b9742571e7b https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/6ff9625b6cb47487bf… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32DnsInfo.c Log Message: ----------- Let lookup account for NULL terminating a WCHAR* The low level `RegQueryValueEx` deals with a char*, but here char* just means some un-interpreted bytes, not a string! If we compile with `-DUNICODE` the un-interpreted bytes will contain a WCHAR* And if we want to properly NULL terminate this WCHAR*, then we need 2 null bytes, not 1! Commit: 32840ac857dbef54e25c61c3e7bb0a9dde5aca5f https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/32840ac857dbef54e2… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32DnsInfo.c Log Message: ----------- Revert to ASCII only version of DnsInfo Rationale: there's no urge in providing localized UNICODE info... That's IP addresses, etc... Eventually, server names could be UNICODE but this seems to be a real mess! The short term goal is to enable compilation with -DUNICODE For longer term, we'll see later. Commit: 475d84cf63fc05ca814fa67016f46e90263d0894 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/475d84cf63fc05ca81… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/plugins/HostWindowPlugin/sqWin32HostWindowPlugin.c Log Message: ----------- iconPath is char*, LoadImage expects a TCHAR* We now interpret iconPath as UTF-8 encoded We convert it to WideChar and call the W version. TODO: for now, do not deal with UNC long filenames... Commit: ef245b6d485b0286f648fda560b0564d812593e5 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ef245b6d485b0286f6… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/plugins/HostWindowPlugin/sqWin32HostWindowPlugin.c Log Message: ----------- And account for the fact that iconPath is not NULL-TERMINATED! Commit: 07ff6a63ab5841ece882f93092d900d36ca371b0 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/07ff6a63ab5841ece8… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- DPRINTF must take a TCHAR*fmt because wvsprintf does! though, vfprintf does not, so reconcile by using _vftprintf from <tchar.h> Commit: dfe4d09597d2c6ed94ff9d1f6305df677bf39067 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/dfe4d09597d2c6ed94… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- NOTIFYICONDATA.szTip maybe a WCHAR* if -DUNICODE so let's use appropriate TCHAR* functions/macros Commit: db33158941b1a8e74224d98ff6e81b2f20e6959c https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/db33158941b1a8e742… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- _DISPLAY_DEVICE.DeviceString may be a WCHAR* if -DUNICODE, we cannot simply sprintf We have to test #ifdef UNICODE, and if so, convert to UTF8 Commit: 46bd992b7633dfc4f96b142104651313b826809c https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/46bd992b7633dfc4f9… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32Prefs.c M platforms/win32/vm/sqWin32Prefs.h Log Message: ----------- VM_VERSION_TEXT is a TCHAR*, we cannot simply fprintf But there is no need for UNICODE in VM_VERSION_TEXT Revert to plain ASCII and rename it VM_VERSION_VERBOSE Commit: c5f207c84ce1a9dca04ff4126e24a7489986c9c5 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/c5f207c84ce1a9dca0… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- iniName, manufacturer and model may be WCHAR* if -DUNICODE Let iniName be WCHAR unconditionally. Get manufacturer and model into a UTF16 buffer, then convert them to UTF8 while at it, protect buffer overrun strcat - > wcsNcat Commit: d9b3927ed91ef60eeca7ec14dd7f48ccb6022cde https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/d9b3927ed91ef60eec… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- Information queried in Registry can be WideChar if -DUNICODE We want to generate an UTF8 report (do we really?). So provide a `RegLookupUTF8String`, so as to make all the queries with W variant, then convert all information to UTF8 While at it, use snprintf instead of sprintf and strncat instead of strcat Commit: 8b14fbfcb43951d7a7f87be1d927fadc7db85007 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/8b14fbfcb43951d7a7… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- Make stderrName and stdoutName be WCHAR* There was a mixture of TCHAR* and char* which could not work with -DUNICODE Who knows, the TempPath might be localized, so go UNICODE... Commit: 643a5e3bc03142a49115588a1b215cf963563e42 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/643a5e3bc03142a491… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32.h M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32Service.c M platforms/win32/vm/sqWin32Window.c Log Message: ----------- Retract support for Windows95 (no you don't dream it's nearly 2019!) I'm very sorry to dilapidate all this historical knowledge, but frankly YAGNI! Even if we are a museum, we can't expose all our art in permanent collections! Note that minimal version is already set to XP (via WINVER:=-D_WIN32_WINNT=0x0501 -DWINVER=0x0501 in Makefile.tools) So we're keeping this stuf for nothing, and now it gets in our way to UNICODE. Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/1893512441e9...64… **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

Test --- please ignore
by Andres Valloud 31 Dec '18

31 Dec '18

Sorry about the noise, happy holidays!

2 1

Build completed: opensmalltalk-vm 0.0.33.bencoman
by AppVeyor 31 Dec '18

31 Dec '18

1 0

Errored: OpenSmalltalk/opensmalltalk-vm#1596 (WIN64_UNICODE - 1893512)
by Travis CI 31 Dec '18

31 Dec '18

Build Update for OpenSmalltalk/opensmalltalk-vm ------------------------------------- Build: #1596 Status: Errored Duration: 55 mins and 57 secs Commit: 1893512 (WIN64_UNICODE) Author: Nicolas Cellier Message: Platform specific knowledge: WIN32_FILE and STD_FILE are mutually exclusive options Therefore, when we define `WIN32_FILE_SUPPORT` we must also define `NO_STD_FILE_SUPPORT` Until now, this knowledge was in build.win*/common/Makefile.* (and a legacy MSVC project in plaftorms/win32/misc/Squeak.dsp - on pourrait réduire la voilure !!!) Since it is easy to forget the second define, lets extend the knowledge in the sqPlatformSpecific.h View the changeset: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/49eff31f0372...18… View the full build log and details: https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/473703547?utm_m… -- You can unsubscribe from build emails from the OpenSmalltalk/opensmalltalk-vm repository going to https://travis-ci.org/account/preferences/unsubscribe?repository=8795279&ut…. Or unsubscribe from *all* email updating your settings at https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notificati…. Or configure specific recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications.

1 0

Build failed: opensmalltalk-vm 0.0.32.bencoman
by AppVeyor 31 Dec '18

31 Dec '18

1 0

Build failed: opensmalltalk-vm 0.0.31.bencoman
by AppVeyor 31 Dec '18

31 Dec '18

1 0

[OpenSmalltalk/opensmalltalk-vm] 627bc5: Use the eventually true UNICODE imageNameT if -DUN...
by GitHub 31 Dec '18

31 Dec '18

Branch: refs/heads/WIN64_UNICODE Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 627bc5e39c3e718de09ddc0293997435b8c73413 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/627bc5e39c3e718de0… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Service.c Log Message: ----------- Use the eventually true UNICODE imageNameT if -DUNICODE using `toUnicode` does not do the right thing: it promotes each UTF8 byte code to short... ... which can hardly work beyond ASCII. Commit: 98fc85ddd56487492a4988ad457dbc2b3fe88397 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/98fc85ddd56487492a… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32ExternalPrims.c Log Message: ----------- interpret pluginName as UTF8 rather than pure ASCII toUnicode is just expanding bytes to short, and re-interpret them UTF16, not regarding original encoding. This works well for pure ASCII, but is not really defined for other encodings. We should prefer using UTF8 by default for all the image-VM string transfer. Commit: 4f6f191613f6bf6cdb989b39e1002cf4938f475c https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/4f6f191613f6bf6cdb… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c M platforms/win32/vm/sqWin32.h M platforms/win32/vm/sqWin32Alloc.c M platforms/win32/vm/sqWin32Main.c M platforms/win32/vm/sqWin32SpurAlloc.c M platforms/win32/vm/sqWin32Utils.c Log Message: ----------- refactor sqMessageBox to void using toUnicode Why to get rid of toUnicode fromUnicode fromSqueak fromSqueak2? - we'd rather use UTF8 everywhere. - and it's the last usage remaining! Now the clients must use a TEXT() macro also for the format. In the future, we could eventually translate the VM messages... Get rid of wsprintf variant which has no character limit and might be prone to buffer overrun. An alternative for supporting both ASCII and WIDE is the _tcs* family in <tchar.h> See https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/vsprintf-v… In order to avoid buffer overrun, prefer a vsnprintf variant Note: I did also update the prototype in sqWin32SpurAlloc, but not the contents. It is dead code, and only there for testing purposes. We'd better remove it! Commit: 545ec0a931166a8374683940f3344e27a110f42c https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/545ec0a931166a8374… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32.h M platforms/win32/vm/sqWin32Utils.c Log Message: ----------- Discard now unused toUnicode fromUnicode fromSqueak fromSqueak2 lstrrchr For UNICODE compatibility, - every String coming from image to the VM should better be interpreted UTF8, and converted to wide String via MultiByteToWideChar() - every String going to the image from the VM should better be converted from Wide string to UTF8 via WideCharToMultiByte() See: https://docs.microsoft.com/en-us/windows/desktop/api/stringapiset/nf-string… https://docs.microsoft.com/en-us/windows/desktop/api/stringapiset/nf-string… Side note: there is also a _tcsrrchr in <tchar.h> at least since visual studio 2015 See https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/strrchr-wc… <tchar.h> is also supported by mingw so if ever we need lstrrchr again, we'll use that. Commit: 2024d4358c14482f4ad861180e412488e25aef7a https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2024d4358c14482f4a… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32.h Log Message: ----------- #if 0 ? YAGNI ! we now use builtin _WIN32 _WIN64 anyway Reminder: even in WIN64, _WIN32 is defined, so the comment was a bit misleading anyway. Commit: e1e83f770aaa7f903509eae7166c858260bd3b7c https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/e1e83f770aaa7f9035… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32.h Log Message: ----------- which SetUpPreferences()? There is no SetUpPreferences()! Commit: 8638522434f52dbacfdf634745ac50e387900ec6 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/8638522434f52dbacf… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-30 (Sun, 30 Dec 2018) Changed paths: M platforms/win32/vm/sqWin32Main.c Log Message: ----------- RegisterWindowMessage takes a TCHAR * Commit: 1893512441e9838d3393896c1fe1abc507775c55 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/1893512441e9838d33… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2018-12-31 (Mon, 31 Dec 2018) Changed paths: M platforms/win32/vm/sqPlatformSpecific.h Log Message: ----------- Platform specific knowledge: WIN32_FILE and STD_FILE are mutually exclusive options Therefore, when we define `WIN32_FILE_SUPPORT` we must also define `NO_STD_FILE_SUPPORT` Until now, this knowledge was in build.win*/common/Makefile.* (and a legacy MSVC project in plaftorms/win32/misc/Squeak.dsp - on pourrait réduire la voilure !!!) Since it is easy to forget the second define, lets extend the knowledge in the sqPlatformSpecific.h Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/49eff31f0372...18… **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

Primitives 34-37 for LargeIntegers bit operations not in use?
by Fabio Niephaus 30 Dec '18

30 Dec '18

Hi all, I just noticed that Squeak-trunk does not use the primitives for LargeIntegers bit operations (#34 to #37). Is there are good reason for this (disabled in the VM?) or is this just a mistake? See: SystemNavigation default browseAllSelect: [:cm | cm primitive between: 34 and: 37] Fabio

2 2

Errored: OpenSmalltalk/opensmalltalk-vm#1595 (WIN64_UNICODE - 49eff31)
by Travis CI 30 Dec '18

30 Dec '18

Build Update for OpenSmalltalk/opensmalltalk-vm ------------------------------------- Build: #1595 Status: Errored Duration: 1 hr, 31 mins, and 9 secs Commit: 49eff31 (WIN64_UNICODE) Author: Nicolas Cellier Message: The pageSize and pageMask are too short on WIN64 Due to this, roundUpToPage is truncating addresses > 2^32 With MSVC, minAddress is > 2^32, and it then takes ages to generate a VirtualAlloc > minAddress (several minutes). View the changeset: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/5f71fce2f077...49… View the full build log and details: https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/builds/473621937?utm_m… -- You can unsubscribe from build emails from the OpenSmalltalk/opensmalltalk-vm repository going to https://travis-ci.org/account/preferences/unsubscribe?repository=8795279&ut…. Or unsubscribe from *all* email updating your settings at https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notificati…. Or configure specific recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications.

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev December 2018