Repository navigation
split function for Data.Conduit.Text #205
Description
Hi, I needed a split function for some log parsing I was doing with Conduit, would you be interested in me adding this to Data.Conduit.Text? If so, I can add tests and remove the Safe dependency that my code is using, then submit a PR.
This is the code I'm currently using; it's modeled closely after lines and I believe lines could be implemented in terms of split.
split :: Monad m => Text -> Conduit Text m Text
split splitText = awaitText T.empty
where
awaitText buf = await >>= maybe (finish buf) (process buf)
finish buf = unless (T.null buf) (yield buf)
process buf text = yieldSplits $ buf `T.append` text
yieldSplits text = do
let splits = T.splitOn splitText text
lastSplit = lastDef T.empty splits
mapM_ yield (initSafe splits)
awaitText lastSplit
Activity
mgsloan commentedon Mar 27, 2015
Just for info / reference, I did something similar when implementing fusion in conduit-combinators: https://github.com/fpco/conduit-combinators/blob/master/Data/Conduit/Combinators.hs#L1889
The main difference is that yours takes a
Textrather thanElement seq -> Bool.MaxGabriel commentedon Mar 27, 2015
Cool @mgsloan. I guess the conduit-combinators version of this would take an
EqSequenceand split its input using that?mgsloan commentedon Mar 27, 2015
Yup, that seems reasonable to me. I'm not seeing an implementation of
splitforEqSequence, though.snoyberg commentedon Mar 28, 2015
I think the code above is subtly broken; if the broken-upon string occurs over a chunk boundary, it won't fire. E.g.:
doesn't look like it would result in the output you'd expect.
snoyberg commentedon Jul 20, 2015
Tracked by PR #207