branchmap: update cache of 'unserved' filter on new changesets The `commitctx` and `addchangegroup` methods of repo upgrade branchcache after completion. This behavior aims to keep the branchcache in sync for read only process as hgweb. See ee317dbfb9d0 for details. Since changelog filtering is used, those calls only update the cache for unfiltered repo. One of no interest for typical read only process like hgweb. Note: By chance in basic case, `repo.unfiltered() == repo.filtered('unserved')` This changesets have the "unserved" cache updated instead. I think this is the only cache that matter for hgweb. We could imagine updating all possible branchcaches instead but: - I'm not sure it would have any benefit impact. It may even increase the odd of all cache being invalidated. - This is more complicated change. So I'm going for updating a single cache only which is already better that updating a cache nobody cares about. This changeset have a few expected impact on the testsuite are different cache are updated. [ original upstream message ]

tests: run with PYTHONHASHSEED=random Python set and dict iteration order is in principle undefined but usually 'quite stable'. Setting PYTHONHASHSEED=random will make the iteration order more random in Python 2.6.8 and 2.7.3 and where it has been backported. This can thus help spot dependencies on undefined behaviour and prevent future problems. [ original upstream message ]

run-tests: exit cleanly if parallel run is interrupted If interrupted while running with "--jobs N", run-tests asynchronously spewed a bunch of output and backtraces from both the master and slave processes, leaving the terminal full of goop. This patch makes it behave more sensibly. [ original upstream message ]

filtering: rename filters to their antonyms Now that changelog filtering is in place, it's become evident that naming the filters according to the set of revs _not_ included in the filtered changelog is confusing. This is especially evident in the collaborative branch cache scheme. This changes the names of the filters to reflect the revs that _are_ included: hidden -> visible unserved -> served mutable -> immutable impactable -> base repoview.filteredrevs is renamed to filterrevs, so that callers read a bit more sensibly, e.g.: filterrevs('visible') # filter revs according to what's visible [ original upstream message ]

Merge with default

test: improve grammar and spelling in branchcache note See: 769eb1b179b5

Merge with stable

hghave: introduce a test (unused) for cvs >= 1.12 [ original upstream message ]

clfilter: enforce hidden changeset globally The dispatch code now enables filtering of "hidden" changesets globally. The filter is installed before command and extension invocation. The `--hidden` switch is now global and disables this filtering for any command. Code in log dedicated to changeset exclusion is removed as this global filtering has the same effect. [ original upstream message ]

branchcache: add note about cache invalidation to test-keyword.t [Should've been included in aff706b3a21c.] --Kevin Bullock <kbullock@ringworld.org> [ original upstream message ]

clfilter: add mutable filtering It filters all mutable changesets, leaving only public changeset unfiltered. This filtering set is expected to be much more stable that the previous one as public changeset are unlikely to disapear. The only official use of this filter is for branchcache. [ original upstream message ]

run-tests.py: fix handling of symlink to the right python Before: a symlink for python in BINDIR was sometimes created, but it was never updated when a different Python was used and it was never removed. An invalid python could thus be left around and used when testing with --local. Now: the symlink is removed when wrong and created when necessary. The mechanism for finding the right name (python or python.exe) also had to be simplified and made more explicit. [ original upstream message ]

Merge with stable

branchmap: allow to use cache of subset Filtered repository are *subset* of unfiltered repository. This means that a filtered branchmap could be use to compute the unfiltered version. And filtered version happen to be subset of each other: - "all() - unserved()" is a subset of "all() - hidden()" - "all() - hidden()" is a subset of "all()" This means that branchmap with "unfiltered" filter can be used as a base for "hidden" branchmap that itself could be used as a base for unfiltered branchmap. unserved < hidden < None This changeset implements this mechanism. If the on disk branchcache is not valid we use the branchcache of the nearest subset as base instead of computing it from scratch. Such fallback can be cascaded multiple time is necessary. Note that both "hidden" and "unserved" set are a bit volatile. We will add more stable filtering in next changesets. This changeset enables collaboration between no filtering and "unserved" filtering. Fixing performance regression introduced by 47f00b0de337 [ original upstream message ]

run-tests.py: don't use console for stdin when running in debug mode Tests would wait for input instead of using non-interactive mode. [ original upstream message ]

tests: make hghave and run-tests exit on unknown feature requirements [ original upstream message ]

Merge with stable

localrepo: filter unknown nodes from the phasecache on destroyed When commit is followed by strip (qrefresh), phasecache contains nodes that were removed from the changelog. Since phasecache is filecached with .hg/store/phaseroots which doesn't change as a result of stripping, we have to filter it manually. If we don't write it immediately, the next time it is read from disk the nodes will be filtered again. That's what happened before, but there's no reason not to write it immediately. The change in test-keyword.t is caused by the above. [ original upstream message ]

Merge with stable

strip: do not update branchcache during strip (issue3745) At this moment, the cache is invalid, and will be thrown away. Later the strip function will call the `localrepo.destroyed` method that will update the branchmap cache. [ original upstream message ]

branchmap: move validity logic in the object itself In several place, We check if a branchcache is still valid regarding the current state of the repository. This changeset puts this logic in a method of the object that can be reused when necessary. A branch map is considered valid whenever it is up to date or a strict subset of the repository state. The change will help making branchcache aware of filtered revision. The change in keyword is expected. the branch cache is actually invalid after the amend. The previous check did not detected it. [ original upstream message ]

Merge with stable

run-tests: fix whitespace nonsense [ original upstream message ]

run-tests: on windows, put correct python at front of PATH The older approach of trying to copy the python executable into the test directory was doomed to fail. There remains one weakness with this approach: if you've run "make local", tests may pick up the wrong extension DLLs from inside the source tree. I don't know why this happens. A reasonable workaround for now is to test either using --local or with a working directory that does not contain built DLLs. [ original upstream message ]

tests: kill daemons early, making breaking at "Accept" prompt safe [ original upstream message ]

run-tests: fix exename on Windows [ original upstream message ]

run-tests: support running tests in parallel on windows Previously, we used os.spawnvp, which doesn't exist on Windows, and isn't needed anyway (the command line begins with an absolute path). We also need a slightly more convoluted way to wait for processes without specifying an order on Windows, as it lacks os.wait. [ original upstream message ]

run-tests: use correct python safely under --jobs [ original upstream message ]

run-tests: check for the correct python when starting [ original upstream message ]

run-tests: use correct python when run with --local [ original upstream message ]

Merge with stable

run-tests: fix an unnoticed check-code violation [ original upstream message ]

run-tests: add a --compiler option Without this option, it is not possible to run the test suite on Windows using mingw's gcc as the compiler. [ original upstream message ]

run-tests: make build command line less intimidating Use a dict for parameters to the format string, instead of a ridiculous number of positional parameters. [ original upstream message ]

run-tests: backout 4a4173519b63 This deleted work in progress to eliminate child processes for -j. [ original upstream message ]

Merge with stable

run-tests.py: remove runqueue's results parameter since it is now a global [ original upstream message ]

run-tests: open child-parent pipes in binary mode Python's pickle is a binary format. [ original upstream message ]

run-tests: add --time option to log times for each test --time also prints out the wall-clock time each test takes in descending order. [ original upstream message ]

run-tests: remove resultslock since it serves no useful purpose Each child process has its own copy of the results dict, so all access to the results dict is serial. [ original upstream message ]

run-tests: use pickle to send results from children to parent JSON would probably be preferable, but it isn't available for Python 2.4 or 2.5. [ original upstream message ]

Merge with stable

run-tests: skip unnecessary operations on the args in the child proces The arguments are already sorted and never empty. [ original upstream message ]

Merge with stable

Merge with default

run-tests: remove dead code for supporting old test scripts [ original upstream message ]

run-tests: handle windows crlf in .py tests again Before af7c6bc48d8d all crlf occurrences in test output on Windows were simply changed to lf. In af7c6bc48d8d it was replaced by more clever handling in the .t test runner ... but the .py runner was forgotten and many .py tests were failing on Windows. The crlf/lf replacement is now reintroduced in the py test runner. [ original upstream message ]

run-tests: make it possible to combine (esc) with (glob) and (re) This makes it possible to combine the annotations ... if done in the right order. [ original upstream message ]

run-tests: alternative way of handling \r on Windows After f71d60da58fb all \r was stripped from output on Windows, and the places where a \r explicitly was expected it was accepted that it was missing. Ugly hack. Instead we now accept that an extra \r might appear at the end of lines on Windows. That is more to the point and less ugly. [ original upstream message ]

amend: add noise in extra to avoid creating obsolescence cycle (issue3664) Obsolescence cycle are bad and should be avoided as much as possible. The current amend implemented touch changeset meta data as few as possible. This make is easy for amend to result in the same node than a precursors. We add some deterministic noise in extra to avoid this. In practice, the hex of the amended changeset is stored in 'amend_source' extra key. [ original upstream message ]

Move commands.inferrepo for code maintenance

Merge with stable

commands: don't infer repo for commands like update (issue2748) Maintain a whitelist of commands to infer the repo for instead. The whitelist contains those commands that take file(s) in the working dir as arguments. [ original upstream message ]

Merge with stable

tests: correctly report a test killed by a signal [ original upstream message ]

Merge with default

Avoid traceback caused by bogus date input (issue3344) Wrap datefilters which split date texts with util.parsedate. We do not abort, as the bogus date must have been given by the user.

Merge with stable

run-tests: allow test output lines to be terminated with \r in addition to \n str.splitlines could not be used in 546c76e5a3e6, but _now_ we would like to have lines with other line endings than \n. Some fine occurences of (esc) markup of \r is replaced with multiple lines ending with '\r (no-eol) (esc)'. That is no win but also no significant loss. This change makes it possible to drop filtercr.py - _that_ is a win. [ original upstream message ]

run-tests: use more explicit criteria for detecting no-eol Preparing for the situation where there could be unsalted lines not terminated with \n. [ original upstream message ]

Merge with stable

subrepo, hghave: use "svn --version --quiet" to determine version number svn --version --quiet is implemented since svn 0.14.1 (August 2002) and prints just the version number, not the long output (21 lines) of "svn --version". Additionally I expect this output format to be more stable, at least it is not changed with different translations. [ original upstream message ]

run-tests: drop unused enumerate [ original upstream message ]

Merge with stable

test-http-branchmap: enable on Windows Tests using "hg serve --daemon" are currently disabled on Windows for lack of proper kill utility. The one shipped with MinGW operates on internal process identifiers and not on the ones recorded by hg serve. Fortunately we can replace most of them by calls to killdaemons.py. This patch is a proof of concept on how to run these tests on Windows. The plan is: - Check test-http-branchmap.t does not fail/hang on the buildbot - Convert all kill utility calls to killdaemons.py calls. - Add a rule in check-code.py to forbid kill calls, or ignore the remaining ones (test-hup.t, etc.). - Possibly drop the 'serve' rule from hghave. The: listening at http://*:$HGPORT1/ line does not appear on Windows because the detached process can no longer write on its parent streams. Grepping hg serve stdout directly causes the parent process to never return and hangs the test. This is a bug, but I have no simple solution and prefer to pay this small price and enable hg serve tests on Windows. [ orginal upstream message ]

Merge with stable

tests: unset variable HG if it is set When hg tries to call itself it can call a different hg executable when this variable is set. Some tests fail when the called hg version is different. [ original upstream message ]

Merge with stable

spelling: destination [ original upstream message ]

killdaemons: take file argument explicitely It makes it easier to use as a generic replacement for kill utility, mostly for Windows tests. [ original upstream message ]

killdaemons: add windows implementation [ original upstream message ]

run-tests: do not duplicate killdaemons() code [ original upstream message ]

pylint, pyflakes: remove unused or duplicate imports [ original upstream message, already done for run-tests.py ]

compat: use open() instead of file() everywhere [ original upstream message only for killdaemons here ]

tests: add killdaemons helper script [ orgininal upstream message ]

Merge with stable

fix trivial spelling errors [ original upstream message ]

Merge with stable

check-code: fix check for trailing whitespace on sh command lines The $ has been without necessary escaping since introduced in c7d23b4ca4ba. [ original upstream message ]

Merge with stable

test-keyword: adapt for Windows [ original upstream message ]

tests: don't use dates before epoch in test-keyword.t Timezone offsets of less than a minute is not shown but can cause displayed dates to be before epoch start - and dates before epoch start is not shown correctly on Windows (see also 4d5b12a5517b). These 'negative' dates could be considered undefined behaviour so we don't care and swap the tests values for timestamp and timezone. [ original upstream message ]

Merge with default

Use ui.formatter for kwfiles output

Merge with stable

strip: incrementally update the branchheads cache after a strip This function augments strip to incrementally update the branchheads cache rather than recompute it from scratch. This speeds up the performance of strip and rebase on repos with long history. The performance optimization only happens if the revisions stripped are all on the same branch and the parents of the stripped revisions are also on that same branch. This adds a few test cases, particularly one that reproduces the extra heads that mpm observed. [ original upstream message ]

Merge with stable

run-tests: replace inline python handling with more native scheme Normally changes in tests are reported like this in diffs: $ cat foo - a + b Using -i mode lets us update tests when the new results are correct and/or populate tests with their output. But with the standard doctest framework, inline Python sections in tests changes instead result in a big failure report that's unhelpful. So here, we replace the doctest calls with a simple compile/eval loop. [ original upstream message - missed change to heredoctest ]

tests: fix readline escape characters in heredoctest.py/test-url.py This fix mirrors the changes made to test-doctest.py in b856071435f7 and 967adcf5910d. Without this change, tests running heredoctest.py can fail on certain versions of OS X when TERM is set to xterm-256color: $ /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -m heredoctest <<EOF > >>> open('b', 'w').write('this' * 1000) > EOF + \x1b[?1034h (no-eol) (esc) A similar problem occurs with test-url.py: $ ./run-tests.py test-url.py --- .../tests/test-url.py.out +++ .../tests/test-url.py.err @@ -0,0 +1 @@ + ERROR: .../test-url.py output changed ! Failed test-url.py: output changed # Ran 1 tests, 0 skipped, 1 failed. [ original upstream message ]

tests: remove temp doctest file when finished running it [ original upstream message ]

tests: add helper script for processing doctests read from stdin Writes stdin to a temp file and doctests it. In the future we might want to spare the temp file and directly call into doctest. Also, with some tweaking it seems possible to adjust the line numbers reported in an error report so they match the ones in the original file. [ original upstream message ]

tests: prepare get-with-headers.py for MSYS get-with-headers.py took the http GET parameter as a command line parameter that had to start with '/'. MSYS on windows will mangle such paths. Instead of applying a workaround everywhere (such as an extra '/') we let get-with-headers.py add the mandatory '/'. That is consistent with the url path handling in the Mercurial url class. A few tests sent 'GET ?cmd=...' which is invalid. They will now send 'GET /?cmd=...'. This will not enable any tests for being run on windows - only remove one reason they were disabled. [ original upstream message ]

tests: hghave outer-repo should be true even if a bad repo is found Only the abort message 'no repository found' is a reliable indication that there is no outer repo. [ original upstream message ]

tests: add missing no-outer-repo requirements Outer repos (if any) will now never be touched by the tests. But it is better to run without any repos around the tmp directory. [ original upstream message ]

Merge with stable

tests: introduce hghave hardlinks Some tests can't be run on FAT filesystems because it doesn't support hardlinks. [ original upstream message ]

tests/hghave: test that the inotify unix socket actually can be created Inotify do not work on FAT filesystems. [ original upstream message ]

tests/hghave: test that a fifo actually can be created on the filesystem Some tests were failing on FAT filesystems. [ original upstream message ]

tests/hghave: consistently use dir='.', prefix=tempprefix for tempfiles Some feature tests were done in the wrong directory and could thus give the wrong result. [ original upstream message ]

tests/hghave: extract hghave.py hghave is degraded to a bare script, moving the functions to hghave.py so they can be shared later on. [ original upstream message ]

Merge with stable

tests: add missing trailing 'cd ..' Many tests didn't change back from subdirectories at the end of the tests ... and they don't have to. The missing 'cd ..' could always be added when another test case is added to the test file. This change do that tests (99.5%) consistently end up in $TESTDIR where they started, thus making it simpler to extend them or move them around. [ original upstream message ]

tests: convert some 'hghave symlink' to #if This will enable some tests for windows. [ original upstream message ]

tests: use the right directory for running hghave from run-tests.py [ original upstream message ]

tests: make .t tests stop immediately if a cd fails This prevents tests from escaping from TESTTMP as a consequence of a failing directory creation. [ original upstream message ]

Merge with stable

tests/hghave: implement #if true / #if false For unconditionally testing / skipping a section. Useful for testing the test infrastructure in test-run-tests.t and for debugging/developing tests. [ original upstream message ]

tests/run-tests: avoid C:/ in arguments MSYS replaces C:/... in arguments with C;... as it interprets the C:/ as a colon separated POSIX path list. The colon is replaced with ; (path separator on Windows) according to http://www.mingw.org/wiki/Posix_path_conversion So we must not replace \ with / for neither $TESTTMP nor $TESTDIR, but we have to keep replacing \ with / for the Popen4 call of function hghave. If we don't do the latter, test-run-tests.t will fail with $ python run-tests.py --local test-run-tests.t --- C:\Users\adi\hgrepos\hg-main\tests\test-run-tests.t +++ C:\Users\adi\hgrepos\hg-main\tests\test-run-tests.t.err @@ -70,6 +70,7 @@ tested #else $ echo skipped + skipped #endif #if false An additional tweak in test-ssh.t is needed that globs away an encoded path, as it can't be translated back to $TESTTMP, because the backslashes in the output have been already encoded as %5C. This patch makes test-ssh.t pass in MSYS on Windows. [ original upstream message ]

hghave: wrap command in 'sh -c "..."' for has_pyflakes() Without this, the has_pyflakes() check always fails in MSYS on Windows. [ original upstream message ]

tests/run-tests: use $TMP on Windows (issue3490) This is just a short-term workaround for that issue. More work needs to be done on scmutil.canonpath & friends. $TMP on Windows is specified to be defined, and it has correct casing, so we can use that as the default dir for tempfile.mkdtemp on Windows. [ original upstream message ]

Merge with stable

tests: introduce c-style conditional sections in .t tests This makes it possible to have conditional sections like: #if windows $ echo foo foo #else $ echo bar bar #endif The directives and skipped sections are treated like comments, so don't interleave them with commands and their output. The parameters to #if are evaluated while preparing the test by passing them over to hghave. Requirements can thus be negated with 'no-' prefix, and multiple requirements must all be true to return true. [ original upstream message ]

run-tests: don't add python lines to expected dict For test input lines of *.t files starting with ' >>> ', the code block for ' >>> ' 609: if l.startswith(' >>> '): # python inlines 610: after.setdefault(pos, []).append(l) was (unsurprisingly) executed, but because there was an "if" instead of an "elif" on the condition "l.startswith(' ... ')", program execution proceeded to line 636 635: elif l.startswith(' '): # results 636: # queue up a list of expected results 637: expected.setdefault(pos, []).append(l[2:]) due to the fact that if l starts with ' >>> ' it also starts with ' '. The net effect was that python command lines in *.t files were (surprisingly) also added to the "expected" dict. This caused no externally observable bad behavior, as the "expected" dict was not consulted for these lines. [ original upstream message ]

run-test: replace backslashes in TESTDIR This may cause troubles in MSYS on Windows. [ original upstream message ]

Merge with default

Update copyleft

Wlock cmdutil.copy wrapper Expanding/shrinking happens outside the wrapped copy function; therefore write lock the repo.

Support commit --amend (issue3471) Include a test as well.

Rename kwt.record attribute to kwt.postcommit A more general descriptive name, as the attribute will be used for commit --amend as well.

Merge with stable

hgext: mark all first-party extensions as such [ original upstream message ]

Merge with default

Intentionally ignore check-code warning about unwrapped ui message

Merge with stable

cleanup: "not x in y" -> "x not in y" [ original upstream message ]

cleanup: replace naked excepts with more specific ones [ original upstream message ]

cleanup: replace hasattr() usage with getattr() in hghave [ original upstream message ]

cleanup: "x != None" -> "x is not None" [ original upstream message ]