fix(service-patterns): match library ids on identifier boundaries
Distilled from #1245 by Andrew Hundt (d15071c9, the match_qn half).
match_qn was a raw strstr(qn, library_id). Short ids fired inside
unrelated words, and a service kind REPLACES the plain CALLS edge, so a
false match both invents a service edge and deletes a real call:
"gin." in plugin. -> route registration; any "/"-prefixed
first argument then minted a Route node
"nconf" in encryptionconfig -> CONFIGURES for every call into the
package
"tonic" in Monotonic -> gRPC; unparseable, so NO edge at all
"express" in expression -> route registration
"get_env" in get_envelope, get_environ -> CONFIGURES
"dio" "surf" "curl" "resty" "phin" "hyper" "treq" "rocket" in studio,
surface, curly, restyle, dolphin, hyperbolic, streq, rocketmq
The rule (qn_hit_on_boundary; every occurrence of an id is tried, not
only the first). A separator is any character that is not an ASCII
letter or digit.
before: start of string | separator | the id itself starts with a
separator ("@trpc/server") | the id starts with an uppercase
letter -- a capital opens a new CamelCase word whatever
precedes it (AsyncHttpClient, IHttpClientFactory, NSURLSession)
after: end of string | separator | the id itself ends in a separator
("gin.") | uppercase letter (GuzzleHttp, FeignClient,
KafkaProducer) | digit (urllib2, Mint.HTTP2, amqp091-go)
Rejected: a lowercase- or digit-initial id glued to a preceding letter
or digit, and any id continued by a lowercase letter. Two deliberate
differences from the upstream hunk: upstream's before-rule needed a
GuzzleHttp table entry and would have dropped IHttpClientFactory and
NSURLSession, and without the digit after-boundary urllib2 and
amqp091-go stop matching. All three match on main today.
No recall is traded for this. No boundary rule can tell "grequests"
from "myrequests", so a library whose own name glues a prefix or suffix
onto another id gets an explicit entry with the kind (and broker) it had
on main -- 47 of them, found mechanically by running main's matcher and
the new one over 173 candidate names and listing every one that went
from a kind to NONE:
HTTP grequests txrequests redaxios gaxios libcurl curlpp curlcpp
hyperlocal guzzlehttp
ASYNC aiokafka pykafka rskafka librdkafka rdkafkacpp aioamqp
amqpstorm pamqp pyamqp amqprs amqpcpp pynats jnats aiomqtt
amqtt hbmqtt umqtt mqttools emqtt rumqtt gomqtt libmosquitto
mosquittopp pubsublite gocelery
CONFIG wgetenv getenvb qgetenv dotenvx phpdotenv
ROUTE_REG apiflask honox fasthttprouter
GRAPHQL gqlparser gqlgenc aiogqlc
Left at NONE on purpose: names that are not clients of the id they
contain and were misclassified on main -- hypercorn, hyperlink,
hyperopt, openresty, gqlalchemy, celeryconfig, kafkacat, gnatsd,
pypubsub. One visible reclassification: hyperium/tonic was HTTP through
"hyper" inside "hyperium" and is now gRPC through "tonic".
RED, production reverted and the final tests kept:
infrascan_service_pattern_match_rejects_ids_inside_words
FAIL tests/test_infrascan.c:105:
svc_case_mismatches(cases) == 28, expected 0 == 0
4 passed, 1 failed
All 28 negatives mismatch on main. The 47 glued-library positives are
RED the other way round: against the boundary rule without the entries.
Each positive uses a name containing no second id, so it binds to its
own entry -- "aiokafka.AIOKafkaProducer" would have passed through
"KafkaProducer" and proven nothing.
GREEN: infrascan 5, pipeline 281, parallel 74, extraction 350,
edge_types_probe 59, route_canon 11, lang_contract 41, cross_repo 8,
registry 64, mcp 318 (4 Windows-only skips). 0 failed.
Effect on real code -- production binaries, main 92abefa3 against this
change. main against main differs by 0 edges and 0 nodes on all three
corpora, so every delta below is the change:
django CONFIGURES 76 -> 74; both calls return as CALLS
(geom.get_envelope, WSGIRequestHandler.get_environ)
typescript CONFIGURES 12,026 -> 12,024; one bogus Route "/*type*/"
gone with its CALLS and HANDLES; 3 calls return as CALLS
kubernetes CONFIGURES 7,962 -> 7,609: all 353 are "nconf" inside
encryptionconfig / authorizationconfig /
authenticationconfig, and all 353 return as CALLS.
8 bogus Routes gone -- os.ReadFile of the serviceaccount
namespace file, t.Run("/TwoWay"), "/tmp/test" -- with 26
CALLS, 2 HANDLES and 21 TESTS edges that pointed at them.
374 plain CALLS restored; 8 of them (validate.Monotonic and
friends) had NO edge at all on main. GRPC_CALLS 60 -> 60.
HTTP_CALLS 398 -> 407: nine calls main swallowed as route
registrations now reach the existing arg_url heuristic like
every other package.
88 of the 408 removed service-derived edges were read at the call site
-- all of django and typescript, 82 of 402 in kubernetes: 88 false
positives, 0 true positives lost. None of the three corpora uses a
glued-name library, so the 47 entries are covered by the unit test only.
Known limits: "_" is a separator, so optical_fiber.len still matches
"fiber." exactly as requests_get matches "requests" by design; a
lowercase id followed by a capital matches (kafkaProducer), which also
admits a dioXide-style name; a user wrapper glued without a separator
(mygetenv) is no longer classified; a glued-name library missing from
the list is NONE until someone adds it.
Co-authored-by: Andrew Hundt <ATHundt@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> M
Martin Vogel committed
c314dea4c1409a3a68fb33706b49e272809b5079
Parent: bf4c476