Add SFTP support #341
Open
Add SFTP support #341
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
intended when using a directory as output path. Previously, it only worked if the path passed ended with a path separator. Also add a call to this function in the example SSH-Client and SSH-Server, so that the option actually works in the examples.
…the config to the path defined in configOutput (that is set with the option -config_output) Also add a call to this function in the example Client, Server and MITM, so that the option is actually working. The call is on purpose at the end of task and not directly after the creation of the config. So it also captures the changes made to the config during the task.
…m file. List.of(..) returns an immutable List. Wrapping it in an ArrayList, allows JAXB to modify the list (overwriting it with the loaded xml content)
… find the subclasses of AbstractExtension (classes that inherits from it)
dynamic actions. Actually only dynamic Key Exchange also receives messages, so we could also implmenet a DynamicSendOnlyMessageAction for Delay Compression and Extension Negotiation. Also added calls to set executed at the end of dynamic actions, so that the action counts as executed even if no message was send (because there was no need for sending or receiving messegaes, e.g. when the opponend does not support the delay compression extension
- Add SshActionFactory.withReceiveOptions() so it is easily possible to add ReceivingOptions to Receiving Actions using the SshActionFactory - use this method to set IGNORE_UNEXPECTED_GLOBAL_REQUESTS_WITHOUT_WANTREPLY for the ChannelOpenConfirmationMessage send by the server. - Change FULL workflow, so that only PTY is started and one ENV set. - Change default perperator of PTY and ENV Channel Requests, so that a Replay is wanted. Keep in mind, you have to set in OpenSSH the option `AcceptEnv PATH` so that the enviroment variable can actually be set (but do this only for testing, its bad practice to allow setiing env variables)
that channel messages can be received while ignoring only channel window adjust messages. - Add addChannelOpenActions() and addChannelRequestSubsystemActions() to WorkflowConfigurationFactory. that is now used for the new workflowTraceType SFTP_INIT. - WorkflowTraceType SFTP_INIT will be extended to include SFTP Handshake, as soon as these are implemented. - Fix update of remote Window Size if window adjust message is handled. Calculation used local window size instead of remote. - Set want replay default to true in preperator for Channel Request Subsystem Messages
… this looks nicer in XML output.
Usefull for testing simple a Global Request I implemented it, because I saw that the Global Request Success and failure was implemendted wrongly using ChannelMessages. Global Requests are now still not fully implemented, but work now better than before :) I also started working on a GlobalRequestManager (and GlobalRequestSucessTcpIPMessage, since it attaches Port information if you do not spcify a port in the request) But I stashed it away, because I could not decide how ti implement the abstraction of the success messages. - Fixes Global Request Success and Failure Message - Adds responce specific data blob to success message (actully could be parsed to port number for tcp ip forward requests)
…equest specific data
…functions. - Created channel manage lists that allow access by local or remote ID (used this naming so there is no misunderstanding to sender and recipient channel IDs) - Reworked all accesses to the old channel list, so it is more correct, and local channel IDs do not split the same list als remote IDs - Added pendingChannel lists, for Channels the client requested, but got no confirmation from the server yet. Server sided, still uses the finished channel list directly via createNewChannelFromDefaults() Added: - MessageSentHandler to ChannelCloseMessages. Before this handler was part of the preperator. But it is more general and usefully in the handler. - MessageSentHandler to ChannelOpenConfimationMessage, for same reason as above. - MessageSentHandler to ChannelWindowAdjustMessage, for same reason as above. - Added a ConfigRemoteChannelId field to the abstract ChannelMessage, so forced remote channels are actually possible. (Keep in mind remote will not know about this channel, if you do not open it) Changes: - [ChannelMessagePreperator] Pick default sender channel first by ConfigLocalChannelId than by channelDefaults.LocalChannelId. If the channel was not found and could not be guessed by earlier channel request, create a new channel (receiver will probably not know this channel) - [ChannelOpenFailureMessageHandler] now only removes a channel from pending channel list (since these are the only channels that should fail to open) - Renamed configSenderChannelId to configLocalChannelId in [ChannelOpenMessage] to avoid confusion - Actually handle the chanelOpenSessionMessage - Force set the channel type "session" when preparing a ChannelOpenSessionMessage - Calculate the Window Adjust Result correctly - All Accesses to the channels should be done through the ChannelManager (preferred using the high level methods)
…lers. Add comment "This should not happen, because WantReply should always be false", where ChannelRequests should not be added to ChannelRequestResponseQueue because these messages should never want reply
- Add tracing for complete payload bytes
- Added expectedDataType, that tells what data is expected as payload. This is set via chanel request messages (TODO: should in future also be set via channel open messages that do not open sessions) - Added Lists: receivedRequestsThatWantReply, sentRequestsThatWantReply. self-explaining. Managed via ChannelManager, Updated correctly via ChannelRequests, and ChannelFailure messages. Used to correlate Channel Success messages with a request. This replaces "channelRequestResponseQueue" in the ChannelManager [ChannelManager] - getChannelByReceivedRequestThatWantReply, replaces guessChannelByReceivedMessages. It does now create a channel only if it does not exist. - Added management methods as explained above [ChannelRequests] - Mainly added adjustContextAfterMessageSent() methods for managing requests lists of the channels correctly [ChannelSuccessMessageHandler] - Does now set the ExpectedDataType correctly, utilizing the first unanswered channel request of the channel for this [CONSTANTS] - Added AUTH_AGENT_OPENSSH_COM ChannelType [Tests] - Updated some tests, so that they "work"
…DataMessages. Implemented DataPackets, that can be used for protocols that are running on Channel Data (inluding AuthAgent). To provide same abstraction as classic SSH packets, I implemented it using AbstractDataPacket and provided PassThroughPacket, that really does nothing, and just pass through the data. This adds unnecessery complexety, but provides abstraction. To provide same abstraction as the classic SSH packet layer I implemented an AbstractDataPacketLayer (incuding Layers: DataPacketLayer and PassThroughPacketLayer). Tho this is not really necessery it only adds more complexity and extra objects that needs to be created. But maybe in future this abstraction is useful. Implemented StringDataMessages so Shell / Exec Commands, can be more easily created. Implemented UnknownDataMessage for data on ChannelDataMessages, that have no assosiated Parser. [SFTP] - Implemented basic Info and Version Message - Added Client Version and Server Version (with default 3) to: Chooser, Config, DefaultChooser, SshContext [ReceiveAction] - Added IGNORE_CHANNEL_DATA_WRAPPER Option, to ignore the ChannelDataMessage, because its data is parsed to a new message [WorkflowConfigurationFactory] - Added addSftpInitActions()
- Added SFTP Extension mechanism
- Extensions can be set via Configuration
- No explicit extention yet implemented. Only Unknown Extension
- Added SFTP File Attributs and Extended Attribtue
- Added Abstract SFTP Request and Response
In contrast to the SSH Messages, I implemended the preperators so, that
if, values already set, these values will not be overwritten / prepared
with defaults. This allows the creation of SFTP Messages with
explicit values without using ModifiableVariable modification.
This could also be implemented for the SSH Messages, if it proofs to be
useful.
Also some formatting, not sure why it did not format on last commit :D
Patterns used:
(",\s*)ArrayConverter.bytesToRawHexString
->
$1() -> ArrayConverter.bytesToRawHexString
(",\s*)ArrayConverter.bytesToHexString
->
$1() -> ArrayConverter.bytesToHexString
(",\s*)backslashEscapeString
->
$1() -> backslashEscapeString
Implemented Abstraction for Requests with Handler and Path. This provides less verbosity in variable name of the path / filename. But provides better abstraction, so changes made to serialization or parsing of path / handler are made in a single place.
Added SFTP Opendir, Mkdir and Rmdir Messages
Renamed mkdir. rmdir and opendir
-> Make perare methods public, so someone could actually only prepare parts of a message, instead of preparing the complete message -> Make parse and serialize methods protected, since there is no real usecase to be able to parse and serialize only parts of a message Make private methods of Perperators / Parsers actually private Add some missing @OverRide annotations
…sages as Unknown Data message. Also set handleTimeoutOnReceiveAsIOException default to false, as it was before my changes.
Add option that prevents dynamically generated actions to be added to the workflow trace. And make it default to not add them, because that is better at the moment for reproducibility. Also added a new dynamic action, that allows to reopen a channel. But I'm not currently using it, because I like the way it is because it is a little bit faster for fuzzing.
…ng no message. I was not sure if the method really does what I want, else I would have used it before.
…nstead of using only random handles from the maintained list.
…red handle index can be set easier.
…ing if v4 parsing fails instead of directly using unknown message.
…data message consume errors if the SFTP version is implemented.
…s no real use anyway without other debug log messages.
…hannel Requests by using abstract class ChannelRequestMessageHandler as super class.
…ee no reason why they should be public. Before my work the visibilities were mixed for the preparator methods, then I changed them all to public. But I now think it would be best to keep them protected, because you should prepare a message completely or just don't.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sorry but this PR, also includes refactoring most of the logging, so that it uses lambda expressions, so that value formatting for logging is done lazy on demand.
Tested only some basics, so I guess fixes will follow as soon as I have done some more testing / implemented the sftp fuzzer.
See commit messages for more details.